CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll
Introduction
Problem Statement
The https://expertiza.ncsu.edu/tree_display/list page contains lists of courses, assignments, and questionnaires. This lists can be very long and takes time to load especially when the instructor also loads public assignments since all of them are loaded and rendered in the list. A more common way to accelerate this is to load the first few records that show up in the screen, and load more records as the user scroll down which is called lazy loading. We would like you to optimize this page by implementing such lazy loading using Jscroll or other plugins since the original tree_list code is written in ReactJS.
Moreover, the tree_display lists have some bugs that you should fix:
1. It always goes to assignment page first, so when a user is editing a course then click browser’s back button. It goes back to the assignment instead of the course tab. The same applies when a user is editing a questionnaire, then click back, it goes back to the assignment, instead of questionnaire. We want you to enable “back” to the correct tab.
2. Secondly, when “Include others' items” checkbox has been checked, then the user is editing an assignment or courses, then goes back. The checkbox is unchecked again. It should remember the last state and load the assignment or courses accordingly.
Task Description
Lazy loading
The following picture is what the page looks like now. We can see from firefox developer tools that getting all data from the database once costs 4303ms which is very slow. Therefore, we need to use infinite scroll to load a certain number of the record at one time e.g., record #1-#10
The instructor page load extremely slow due to too much backend Database query ran before loading the front end. We could analyze both frontend and backend of this page to figure out a best solution to improve the performance of this page. The backend part could try to translate more than a thousand SQLs to just a few by Query modification. The frontend can use infinite scroll to reduce the render consumption.
At first, we tried to solved the problem using jscroll. However, it did not make any difference and there were only a few examples and documentation online. Those resources did not help us find the bug. Therefore, because the instructor said we do not have to use jscroll to implement infinite scroll, we decided to use a react plugin called "react-infinite" instead.
Bugs
After we decided to move to another project, Prof. Gehringer mentioned two minor bugs in tree_display/list page.
Bug1
In "Questionnaires" table, after the user click one row to expand the table, he cannot click again to close it. It would raise an error "TypeError: React.addons is undefined"
Bug2
"Others' Public Assignments" list is very long and unorganized. We need to organize it first by "Instructor" and then by "Creation Date".
Plan
Steps
The majority of the logic behind the tree display page was written using ReactJS in expertiza/assets/javascript/tree_display.jsx. So we need to take the following steps:
- 1. Install the dependencies we need to add infinite scroll functionality to ReactJS.
- 2. Since there is no easy way to implement infinite scroll in a table tag of html, we should figure out a way to refactor the whole structure of tree_list.
- 3. Add react-infinite to the tree_list.
- 4. Modify the corresponding functions in assignment, course, questionnaire controllers to support lazy loading. For example, we may need to add "LIMIT" to the statement which query data from the database.
Files to be Modified
app/assets/javascript/application.js app/assets/javascript/tree_display.jsx
app/controllers/assignment_questionnaire_controller.rb app/controllers/assignments_controller.rb app/controllers/course_controller.rb
Tools
react-infinite https://github.com/seatgeek/react-infinite
react-infinite is A browser-ready efficient scrolling container based on UITableView.
Here's a demo http://chairnerd.seatgeek.com/react-infinite-a-browser-ready-efficient-scrolling-container-based-on-uitableview/
Previous Test Plan
- When a user first navigate to the page, there are 10 records and he can scroll down to see more records.
- When a user edits a course, he will go back to course tab after clicking browser's back button .
- When a user edits a assignment, he will go back to assignment tab after clicking browser's back button .
- When a user edits a questionnaire, he will go back to questionnaire tab after clicking browser's back button.
- When "Include others' items" checkbox has been checked, then the user is editing an assignment or courses, then goes back, the checkbox remains checked.
- When "Include others' items" checkbox has not been checked, then the user is editing an assignment or courses, then goes back, the checkbox remains unchecked.
Previous Attempt
While we are working on this project, we have tried multiple approaches, but nothing had worked out.
The first one is jscroll, we had load jscroll into vendor directory using bower, and able to required in the project, but it turns out that jscroll doesn’t work in ReactJS.
Thus, we tried to find third party React Plugins to achieve the function of infinite scroll.
The first React Plugin is react-infinite, We were also able to load react-infinite into vendor directory using bower, and has no error occurred while adding to the react component.
add dependency "react-infinite" to bower.json
"react-infinite": "*"
run setup.sh to load "react-infinite" into /vendor/ modify "app/assets/javascript/application.js" to require the plugin
//= require react-infinite
add "react-infinite" tags to "app/assets/javascript/tree_display.jsx"
<react-infinite> </react-infinite>
However, it also didn’t work out because of the main content is html table.
After that, we have asked this problem on the stackoverflow. People recommended another plugin called react-infinite-scroller, this plugin couldn’t find by bower package manager, so we have to load this package by another package manager like webpacker, but webpacker needs Rails 5 to load React, however Expertiza is based on Rails 4.2.6. We’ve also tried sprockets but we were unable to import.
Our last try is https://infinite-scroll.com/ https://github.com/metafizzy/infinite-scroll
componentWillUpdate: function() {
console.log(1111);
var $container = $('.scroll').infiniteScroll({
// path is required, hack with {{#}}
path: 'page{{#}}',
loadOnScroll: false,
scrollThreshold: 400
});
$container.on( 'scrollThreshold.infiniteScroll', function() {
console.log(2222);
$container.append(<ContentTableRow
key={1}
id={1}
name={1}
directory={1}
instructor={1}
institution={1}
creation_date={1}
updated_date={1}
actions={1}
is_available={1}
course_id={1}
max_team_size={1}
is_intelligent={1}
require_quiz={1}
dataType={1}
//this is just a hack. All current users courses are marked as private during fetch for display purpose.
private={1}
allow_suggestions={1}
has_topic={1}
rowClicked={1}
newParams={1}
/>);
})
},
But we still stuck at this bug.
TypeError: instance._init is not a function.
This project turns out to be really complicated to implement. So we asked to switch the project to implement Rspec integration test for the users_controller.rb
Implementation & Test Plan
Bug1
Because React.addons cannot be used here, I changed it to the function of JS.
var index = this.state.expandedRow.indexOf(id)
if (index > -1) {
var list = this.state.expandedRow;
list.splice(index,1);
this.setState({
expandedRow: list
})
}
Bug2
I sorted the list before the data is returned to the frontend in the tree_display_controller.rb. The list is sorted first in ascending order of 'instructor' and then in the descending order of 'creation_date' which requires 'creation_date' to be converted to timestamp.
res['Assignments'] = res['Assignments'].sort_by {|x| [x['instructor'],-1*(x['creation_date'].to_i)]}
users_controller.rb Test
Our new project is to write Rspec Test for the users_controller.rb.
Here is our code.
require 'pry'
describe UsersController do
let(:admin) {build(:admin)}
let(:superadmin) {build(:superadmin)}
let(:instructor) {build(:instructor, id: 6)}
let(:instructor2) {build(:instructor, id: 66)}
let(:ta) {build(:teaching_assistant, id: 8)}
let(:student) {build(:student)}
before(:each) do
allow(User).to receive(:find_by_name).with('instructor6').and_return(instructor)
allow(User).to receive(:find_by_name).with('instructor66').and_return(instructor2)
allow(User).to receive(:find).with('1').and_return(instructor2)
end
describe '#action_allowed?' do
context 'when params action is request_new' do
it 'allows certain action' do
controller.params = {action: 'request_new'}
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when params action is request_user_create' do
it 'allows certain action' do
controller.params = {action: 'request_user_create'}
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when params action is review' do
it 'allows certain action' do
controller.params = {action: 'review'}
stub_current_user(superadmin, superadmin.role.name, superadmin.role)
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when params action is keys' do
it 'allows certain action' do
controller.params = {action: 'keys'}
stub_current_user(student, student.role.name, student.role)
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when the current role is administrator and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(admin, admin.role.name, admin.role)
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when the current role is superadministrator and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(superadmin, superadmin.role.name, superadmin.role)
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when the current role is instructor and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(instructor, instructor.role.name, instructor.role)
expect(controller.send(:action_allowed?)).to be true
end
end
context 'when the current role is ta and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(ta, ta.role.name, ta.role)
expect(controller.send(:action_allowed?)).to be true
end
end
end
describe '#index' do
context 'when the current user is student' do
it 'redirect to the home page' do
stub_current_user(student, student.role.name, student.role)
get :index
expect(response).to redirect_to('/')
end
end
context 'when the current user is not student' do
it 'redirect to /users/list page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :index
expect(response).to render_template(:list)
end
end
end
describe '#list' do
it 'render list' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :list
expect(response).to render_template(:list)
end
end
describe '#list_pending_requested' do
it 'render list pending' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :list_pending_requested
expect(response).to render_template(:list_pending_requested)
end
end
describe '#new' do
it 'creates a new user and renders users#new page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :new
expect(response).to render_template(:new)
expect(assigns(:user)).to be_a_new(User)
end
end
describe '#request_new' do
context 'when the current user is not student' do
it 'creates a new user and renders users#new page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :new
expect(response).to render_template(:new)
expect(assigns(:user)).to be_a_new(User)
end
end
end
describe '#show_selection' do
context 'when user is not nil and parent id is nil' do
it 'redirect to /users/show' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show_selection, {user: {name: 'instructor6'}}
expect(response).to render_template(:show)
end
end
end
describe '#show' do
context 'when params id is nil' do
it 'redirect to /tree_display/drill' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show, id: nil
expect(response).to redirect_to('/tree_display/drill')
end
end
context 'when params id is not nil' do
it 'render show' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show, id: 1
expect(response).to render_template(:show)
end
end
end
describe '#edit' do
it 'can find user' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :edit, id: 1
expect(response).to render_template(:edit)
end
end
end
Test Result
The test result is as below:
