Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T17:03:58Z

Xli76:

== Introduction ==

===Problem Statement ===
The https://expertiza.ncsu.edu/tree_display/list page contains lists of courses, assignments, and questionnaires. This lists can be very long and takes time to load especially when the instructor also loads public assignments since all of them are loaded and rendered in the list. A more common way to accelerate this is to load the first few records that show up in the screen, and load more records as the user scroll down which is called lazy loading.
We would like you to optimize this page by implementing such lazy loading using Jscroll or other plugins since the original tree_list code is written in ReactJS.

Moreover, the tree_display lists have some bugs that you should fix:

1. It always goes to assignment page first, so when a user is editing a course then click browser’s back button. It goes back to the assignment instead of the course tab. The same applies when a user is editing a questionnaire, then click back, it goes back to the assignment, instead of questionnaire. We want you to enable “back” to the correct tab.

2. Secondly, when “Include others' items” checkbox has been checked, then the user is editing an assignment or courses, then goes back. The checkbox is unchecked again. It should remember the last state and load the assignment or courses accordingly.

== Task Description ==
=== Lazy loading ===
The following picture is what the page looks like now. We can see from firefox developer tools that getting all data from the database once costs 4303ms which is very slow. Therefore, we need to use infinite scroll to load a certain number of the record at one time e.g., record #1-#10

[[File:Dfgjhdfj333.png]]

The instructor page load extremely slow due to too much backend Database query ran before loading the front end. We could analyze both frontend and backend of this page to figure out a best solution to improve the performance of this page. The backend part could try to translate more than a thousand SQLs to just a few by Query modification. The frontend can use infinite scroll to reduce the render consumption.

At first, we tried to solved the problem using jscroll. However, it did not make any difference and there were only a few examples and documentation online. Those resources did not help us find the bug. Therefore, because the instructor said we do not have to use jscroll to implement infinite scroll, we decided to use a react plugin called "react-infinite" instead.

=== Minor bugs ===
After we decided to move to another project, Prof. Gehringer mentioned two minor bugs in tree_display/list page.

==== Minor bug1 ====
[[File:Najsbfdfdfdf.PNG]]

In "Questionnaires" table, after the user click one row to expand the table, he cannot click again to close it. It would raise an error "TypeError: React.addons is undefined"

==== Minor bug2 ====

"Others' Public Assignments" list is very long and unorganized. We need to organize it first by "Instructor" and then by "Creation Date".

== Plan ==
=== Steps ===
The majority of the logic behind the tree display page was written using ReactJS in expertiza/assets/javascript/tree_display.jsx. So we need to take the following steps:
* 1. Install the dependencies we need to add infinite scroll functionality to ReactJS.
* 2. Since there is no easy way to implement infinite scroll in a table tag of html, we should figure out a way to refactor the whole structure of tree_list.
* 3. Add react-infinite to the tree_list.
* 4. Modify the corresponding functions in assignment, course, questionnaire controllers to support lazy loading. For example, we may need to add "LIMIT" to the statement which query data from the database.

=== Files to be Modified ===
app/assets/javascript/application.js
app/assets/javascript/tree_display.jsx

app/controllers/assignment_questionnaire_controller.rb
app/controllers/assignments_controller.rb
app/controllers/course_controller.rb

=== Tools ===
react-infinite
https://github.com/seatgeek/react-infinite

react-infinite is A browser-ready efficient scrolling container based on UITableView.

Here's a demo http://chairnerd.seatgeek.com/react-infinite-a-browser-ready-efficient-scrolling-container-based-on-uitableview/

== Previous Test Plan ==
<ol>
<li>When a user first navigate to the page, there are 10 records and he can scroll down to see more records.
<li>When a user edits a course, he will go back to course tab after clicking browser's back button .
<li>When a user edits a assignment, he will go back to assignment tab after clicking browser's back button .
<li>When a user edits a questionnaire, he will go back to questionnaire tab after clicking browser's back button.
<li>When "Include others' items" checkbox has been checked, then the user is editing an assignment or courses, then goes back, the checkbox remains checked.
<li>When "Include others' items" checkbox has not been checked, then the user is editing an assignment or courses, then goes back, the checkbox remains unchecked.
</ol>

== Previous Attempt ==
While we are working on this project, we have tried multiple approaches, but nothing had worked out.

The first one is jscroll, we had load jscroll into vendor directory using bower, and able to required in the project, but it turns out that jscroll doesn’t work in ReactJS.

Thus, we tried to find third party React Plugins to achieve the function of infinite scroll.

The first React Plugin is react-infinite, We were also able to load react-infinite into vendor directory using bower, and has no error occurred while adding to the react component.

add dependency "react-infinite" to bower.json
"react-infinite": "*"
run setup.sh to load "react-infinite" into /vendor/
modify "app/assets/javascript/application.js" to require the plugin
//= require react-infinite
add "react-infinite" tags to "app/assets/javascript/tree_display.jsx"
<react-infinite>
</react-infinite>

However, it also didn’t work out because of the main content is html table.

After that, we have asked this problem on the stackoverflow. People recommended another plugin called react-infinite-scroller, this plugin couldn’t find by bower package manager, so we have to load this package by another package manager like webpacker, but webpacker needs Rails 5 to load React, however Expertiza is based on Rails 4.2.6. We’ve also tried sprockets but we were unable to import.

Our last try is https://infinite-scroll.com/ https://github.com/metafizzy/infinite-scroll

componentWillUpdate: function() {
console.log(1111);
var $container = $('.scroll').infiniteScroll({
// path is required, hack with {{#}}
path: 'page{{#}}',
loadOnScroll: false,
scrollThreshold: 400
});
$container.on( 'scrollThreshold.infiniteScroll', function() {
console.log(2222);
$container.append(<ContentTableRow
key={1}
id={1}
name={1}
directory={1}
instructor={1}
institution={1}
creation_date={1}
updated_date={1}
actions={1}
is_available={1}
course_id={1}
max_team_size={1}
is_intelligent={1}
require_quiz={1}
dataType={1}
//this is just a hack. All current users courses are marked as private during fetch for display purpose.
private={1}
allow_suggestions={1}
has_topic={1}
rowClicked={1}
newParams={1}
/>);
})
},

But we still stuck at this bug.

TypeError: instance._init is not a function.

This project turns out to be really complicated to implement. So we asked to switch the project to implement Rspec integration test for the users_controller.rb

== Implementation & Test Plan ==

=== Minor bug1 ===

Because React.addons cannot be used here, I changed it to the function of JS.

var index = this.state.expandedRow.indexOf(id)
if (index > -1) {
var list = this.state.expandedRow;
list.splice(index,1);
this.setState({
expandedRow: list
})
}

=== Minor bug2 ===

I sorted the list before the data is returned to the frontend in the tree_display_controller.rb. The list is sorted first in ascending order of 'instructor' and then in the descending order of 'creation_date' which requires 'creation_date' to be converted to timestamp.

res['Assignments'] = res['Assignments'].sort_by {|x| [x['instructor'],-1*(x['creation_date'].to_i)]}

=== users_controller.rb Test ===
Our new project is to write Rspec Test for the users_controller.rb.

Here is our code.
require 'pry'

describe UsersController do
let(:admin) {build(:admin)}
let(:superadmin) {build(:superadmin)}
let(:instructor) {build(:instructor, id: 6)}
let(:instructor2) {build(:instructor, id: 66)}
let(:ta) {build(:teaching_assistant, id: 8)}
let(:student) {build(:student)}
before(:each) do
allow(User).to receive(:find_by_name).with('instructor6').and_return(instructor)
allow(User).to receive(:find_by_name).with('instructor66').and_return(instructor2)
allow(User).to receive(:find).with('1').and_return(instructor2)
end

describe '#action_allowed?' do
context 'when params action is request_new' do
it 'allows certain action' do
controller.params = {action: 'request_new'}
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when params action is request_user_create' do
it 'allows certain action' do
controller.params = {action: 'request_user_create'}
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when params action is review' do
it 'allows certain action' do
controller.params = {action: 'review'}
stub_current_user(superadmin, superadmin.role.name, superadmin.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when params action is keys' do
it 'allows certain action' do
controller.params = {action: 'keys'}
stub_current_user(student, student.role.name, student.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is administrator and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(admin, admin.role.name, admin.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is superadministrator and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(superadmin, superadmin.role.name, superadmin.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is instructor and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(instructor, instructor.role.name, instructor.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is ta and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(ta, ta.role.name, ta.role)
expect(controller.send(:action_allowed?)).to be true
end
end
end

describe '#index' do
context 'when the current user is student' do
it 'redirect to the home page' do
stub_current_user(student, student.role.name, student.role)
get :index
expect(response).to redirect_to('/')
end
end

context 'when the current user is not student' do
it 'redirect to /users/list page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :index
expect(response).to render_template(:list)
end
end
end

describe '#list' do
it 'render list' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :list
expect(response).to render_template(:list)
end
end
describe '#list_pending_requested' do
it 'render list pending' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :list_pending_requested
expect(response).to render_template(:list_pending_requested)
end
end

describe '#new' do
it 'creates a new user and renders users#new page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :new
expect(response).to render_template(:new)
expect(assigns(:user)).to be_a_new(User)
end
end

describe '#request_new' do
context 'when the current user is not student' do
it 'creates a new user and renders users#new page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :new
expect(response).to render_template(:new)
expect(assigns(:user)).to be_a_new(User)
end
end
end

describe '#show_selection' do
context 'when user is not nil and parent id is nil' do
it 'redirect to /users/show' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show_selection, {user: {name: 'instructor6'}}
expect(response).to render_template(:show)
end
end
end

describe '#show' do
context 'when params id is nil' do
it 'redirect to /tree_display/drill' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show, id: nil
expect(response).to redirect_to('/tree_display/drill')
end
end

context 'when params id is not nil' do
it 'render show' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show, id: 1
expect(response).to render_template(:show)
end
end
end

describe '#edit' do
it 'can find user' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :edit, id: 1
expect(response).to render_template(:edit)
end
end
end

=== Test Result ===
The test result is as below:

[[File:Sadasd.PNG]]

== Reference ==
<ol>
<li> [http://jscroll.com/ jscroll]
<li> [https://github.com/seatgeek/react-infinite react-infinite]
</ol>

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T17:03:31Z

Xli76: /* Test Result */

== Introduction ==

===Problem Statement ===
The https://expertiza.ncsu.edu/tree_display/list page contains lists of courses, assignments, and questionnaires. This lists can be very long and takes time to load especially when the instructor also loads public assignments since all of them are loaded and rendered in the list. A more common way to accelerate this is to load the first few records that show up in the screen, and load more records as the user scroll down which is called lazy loading.
We would like you to optimize this page by implementing such lazy loading using Jscroll or other plugins since the original tree_list code is written in ReactJS.

Moreover, the tree_display lists have some bugs that you should fix:

1. It always goes to assignment page first, so when a user is editing a course then click browser’s back button. It goes back to the assignment instead of the course tab. The same applies when a user is editing a questionnaire, then click back, it goes back to the assignment, instead of questionnaire. We want you to enable “back” to the correct tab.

2. Secondly, when “Include others' items” checkbox has been checked, then the user is editing an assignment or courses, then goes back. The checkbox is unchecked again. It should remember the last state and load the assignment or courses accordingly.

== Task Description ==
=== Lazy loading ===
The following picture is what the page looks like now. We can see from firefox developer tools that getting all data from the database once costs 4303ms which is very slow. Therefore, we need to use infinite scroll to load a certain number of the record at one time e.g., record #1-#10

[[File:Dfgjhdfj333.png]]

The instructor page load extremely slow due to too much backend Database query ran before loading the front end. We could analyze both frontend and backend of this page to figure out a best solution to improve the performance of this page. The backend part could try to translate more than a thousand SQLs to just a few by Query modification. The frontend can use infinite scroll to reduce the render consumption.

At first, we tried to solved the problem using jscroll. However, it did not make any difference and there were only a few examples and documentation online. Those resources did not help us find the bug. Therefore, because the instructor said we do not have to use jscroll to implement infinite scroll, we decided to use a react plugin called "react-infinite" instead.

=== Minor bugs ===
After we decided to move to another project, Prof. Gehringer mentioned two minor bugs in tree_display/list page.

==== Minor bug1 ====
[[File:Najsbfdfdfdf.PNG]]

In "Questionnaires" table, after the user click one row to expand the table, he cannot click again to close it. It would raise an error "TypeError: React.addons is undefined"

==== Minor bug2 ====

"Others' Public Assignments" list is very long and unorganized. We need to organize it first by "Instructor" and then by "Creation Date".

== Plan ==
=== Steps ===
The majority of the logic behind the tree display page was written using ReactJS in expertiza/assets/javascript/tree_display.jsx. So we need to take the following steps:
* 1. Install the dependencies we need to add infinite scroll functionality to ReactJS.
* 2. Since there is no easy way to implement infinite scroll in a table tag of html, we should figure out a way to refactor the whole structure of tree_list.
* 3. Add react-infinite to the tree_list.
* 4. Modify the corresponding functions in assignment, course, questionnaire controllers to support lazy loading. For example, we may need to add "LIMIT" to the statement which query data from the database.

=== Files to be Modified ===
app/assets/javascript/application.js
app/assets/javascript/tree_display.jsx

app/controllers/assignment_questionnaire_controller.rb
app/controllers/assignments_controller.rb
app/controllers/course_controller.rb

=== Tools ===
react-infinite
https://github.com/seatgeek/react-infinite

react-infinite is A browser-ready efficient scrolling container based on UITableView.

Here's a demo http://chairnerd.seatgeek.com/react-infinite-a-browser-ready-efficient-scrolling-container-based-on-uitableview/

== Previous Test Plan ==
<ol>
<li>When a user first navigate to the page, there are 10 records and he can scroll down to see more records.
<li>When a user edits a course, he will go back to course tab after clicking browser's back button .
<li>When a user edits a assignment, he will go back to assignment tab after clicking browser's back button .
<li>When a user edits a questionnaire, he will go back to questionnaire tab after clicking browser's back button.
<li>When "Include others' items" checkbox has been checked, then the user is editing an assignment or courses, then goes back, the checkbox remains checked.
<li>When "Include others' items" checkbox has not been checked, then the user is editing an assignment or courses, then goes back, the checkbox remains unchecked.
</ol>

== Reference ==
<ol>
<li> [http://jscroll.com/ jscroll]
<li> [https://github.com/seatgeek/react-infinite react-infinite]
</ol>

== Previous Attempt ==
While we are working on this project, we have tried multiple approaches, but nothing had worked out.

The first one is jscroll, we had load jscroll into vendor directory using bower, and able to required in the project, but it turns out that jscroll doesn’t work in ReactJS.

Thus, we tried to find third party React Plugins to achieve the function of infinite scroll.

The first React Plugin is react-infinite, We were also able to load react-infinite into vendor directory using bower, and has no error occurred while adding to the react component.

add dependency "react-infinite" to bower.json
"react-infinite": "*"
run setup.sh to load "react-infinite" into /vendor/
modify "app/assets/javascript/application.js" to require the plugin
//= require react-infinite
add "react-infinite" tags to "app/assets/javascript/tree_display.jsx"
<react-infinite>
</react-infinite>

However, it also didn’t work out because of the main content is html table.

After that, we have asked this problem on the stackoverflow. People recommended another plugin called react-infinite-scroller, this plugin couldn’t find by bower package manager, so we have to load this package by another package manager like webpacker, but webpacker needs Rails 5 to load React, however Expertiza is based on Rails 4.2.6. We’ve also tried sprockets but we were unable to import.

Our last try is https://infinite-scroll.com/ https://github.com/metafizzy/infinite-scroll

componentWillUpdate: function() {
console.log(1111);
var $container = $('.scroll').infiniteScroll({
// path is required, hack with {{#}}
path: 'page{{#}}',
loadOnScroll: false,
scrollThreshold: 400
});
$container.on( 'scrollThreshold.infiniteScroll', function() {
console.log(2222);
$container.append(<ContentTableRow
key={1}
id={1}
name={1}
directory={1}
instructor={1}
institution={1}
creation_date={1}
updated_date={1}
actions={1}
is_available={1}
course_id={1}
max_team_size={1}
is_intelligent={1}
require_quiz={1}
dataType={1}
//this is just a hack. All current users courses are marked as private during fetch for display purpose.
private={1}
allow_suggestions={1}
has_topic={1}
rowClicked={1}
newParams={1}
/>);
})
},

But we still stuck at this bug.

TypeError: instance._init is not a function.

This project turns out to be really complicated to implement. So we asked to switch the project to implement Rspec integration test for the users_controller.rb

== Implementation & Test Plan ==

=== Minor bug1 ===

Because React.addons cannot be used here, I changed it to the function of JS.

var index = this.state.expandedRow.indexOf(id)
if (index > -1) {
var list = this.state.expandedRow;
list.splice(index,1);
this.setState({
expandedRow: list
})
}

=== Minor bug2 ===

I sorted the list before the data is returned to the frontend in the tree_display_controller.rb. The list is sorted first in ascending order of 'instructor' and then in the descending order of 'creation_date' which requires 'creation_date' to be converted to timestamp.

res['Assignments'] = res['Assignments'].sort_by {|x| [x['instructor'],-1*(x['creation_date'].to_i)]}

=== users_controller.rb Test ===
Our new project is to write Rspec Test for the users_controller.rb.

Here is our code.
require 'pry'

describe UsersController do
let(:admin) {build(:admin)}
let(:superadmin) {build(:superadmin)}
let(:instructor) {build(:instructor, id: 6)}
let(:instructor2) {build(:instructor, id: 66)}
let(:ta) {build(:teaching_assistant, id: 8)}
let(:student) {build(:student)}
before(:each) do
allow(User).to receive(:find_by_name).with('instructor6').and_return(instructor)
allow(User).to receive(:find_by_name).with('instructor66').and_return(instructor2)
allow(User).to receive(:find).with('1').and_return(instructor2)
end

describe '#action_allowed?' do
context 'when params action is request_new' do
it 'allows certain action' do
controller.params = {action: 'request_new'}
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when params action is request_user_create' do
it 'allows certain action' do
controller.params = {action: 'request_user_create'}
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when params action is review' do
it 'allows certain action' do
controller.params = {action: 'review'}
stub_current_user(superadmin, superadmin.role.name, superadmin.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when params action is keys' do
it 'allows certain action' do
controller.params = {action: 'keys'}
stub_current_user(student, student.role.name, student.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is administrator and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(admin, admin.role.name, admin.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is superadministrator and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(superadmin, superadmin.role.name, superadmin.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is instructor and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(instructor, instructor.role.name, instructor.role)
expect(controller.send(:action_allowed?)).to be true
end
end

context 'when the current role is ta and params action is something else' do
it 'allows certain action' do
controller.params = {action: 'view'}
stub_current_user(ta, ta.role.name, ta.role)
expect(controller.send(:action_allowed?)).to be true
end
end
end

describe '#index' do
context 'when the current user is student' do
it 'redirect to the home page' do
stub_current_user(student, student.role.name, student.role)
get :index
expect(response).to redirect_to('/')
end
end

context 'when the current user is not student' do
it 'redirect to /users/list page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :index
expect(response).to render_template(:list)
end
end
end

describe '#list' do
it 'render list' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :list
expect(response).to render_template(:list)
end
end
describe '#list_pending_requested' do
it 'render list pending' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :list_pending_requested
expect(response).to render_template(:list_pending_requested)
end
end

describe '#new' do
it 'creates a new user and renders users#new page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :new
expect(response).to render_template(:new)
expect(assigns(:user)).to be_a_new(User)
end
end

describe '#request_new' do
context 'when the current user is not student' do
it 'creates a new user and renders users#new page' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :new
expect(response).to render_template(:new)
expect(assigns(:user)).to be_a_new(User)
end
end
end

describe '#show_selection' do
context 'when user is not nil and parent id is nil' do
it 'redirect to /users/show' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show_selection, {user: {name: 'instructor6'}}
expect(response).to render_template(:show)
end
end
end

describe '#show' do
context 'when params id is nil' do
it 'redirect to /tree_display/drill' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show, id: nil
expect(response).to redirect_to('/tree_display/drill')
end
end

context 'when params id is not nil' do
it 'render show' do
stub_current_user(instructor, instructor.role.name, instructor.role)
get :show, id: 1
expect(response).to render_template(:show)
end
end
end

describe '#edit' do
it 'can find user' do
stub_current_user(instructor, instructor.role.name, instructor.role)
@request.session['user'] = instructor
get :edit, id: 1
expect(response).to render_template(:edit)
end
end
end

=== Test Result ===
The test result is as below:

[[File:Sadasd.PNG]]

File:Sadasd.PNG

2017-12-03T17:03:11Z

Xli76:

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T16:56:38Z

Xli76: /* Implementation & Test Plan */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T16:49:43Z

Xli76: /* Implementation & Test Plan */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T16:48:33Z

Xli76: /* Previous Attempt */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T16:47:28Z

Xli76: /* Implementation & Test Plan */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T16:37:46Z

Xli76: /* Minor bug1 */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-12-03T16:20:04Z

Xli76: /* Task Description */

File:Najsbfdfdfdf.PNG

2017-12-03T16:18:54Z

Xli76: uploaded a new version of "File:Najsbfdfdfdf.PNG"

File:Najsbfdfdfdf.PNG

2017-12-03T16:18:16Z

Xli76:

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-19T15:17:08Z

Xli76: /* Implementation */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-19T15:16:53Z

Xli76: /* Implementation */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:44:33Z

Xli76: /* Tools */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:44:14Z

Xli76: /* Tools */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:39:46Z

Xli76: /* Test Plan */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:36:16Z

Xli76: /* Minor bug1 */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:30:33Z

Xli76: /* Implementation */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:15:25Z

Xli76: /* Minor bug2 */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T02:09:41Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T01:55:18Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T00:29:09Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T00:17:14Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T00:16:10Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-15T00:06:54Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-14T22:22:47Z

Xli76: /* Task Description */

File:Dfgjhdfj333.png

2017-11-14T22:21:01Z

Xli76: uploaded a new version of "File:Dfgjhdfj333.png"

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-08T03:28:00Z

Xli76: /* Information about jScroll */

CSC/ECE 517 Fall 2017

2017-11-08T00:53:08Z

Xli76:

'''Design Project Documentation'''
*[[CSC/ECE 517 Fall 2017/E17A2 Lightweight Badging System]]
*[[CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them]]
*[[CSC/ECE 517 Fall 2017/E1793. Help students find teams to join]]
*[[CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment]]
*[[CSC/ECE 517 Fall 2017/E17A0 Team-based reviewing]]
*[[CSC/ECE 517 Fall 2017/E1793. Help students find teams to join_Team1964]]
*[[CSC/ECE 517 Fall 2017/E1791. Track the time that students look at the other submissions - logging improvement]]
*[[CSC/ECE 517 Fall 2017/E1799 Improve self-review Link peer review & self-review to derive grades]]
*[[CSC/ECE_517_Fall_2017/E1797_Timestamps_for_students_submissions]]
*[[CSC/ECE_517_Fall_2017/E17A1|CSC/ECE_517_Fall_2017/E17A1 - Let experts as well as students do reviews]]
*[[CSC/ECE_517_Fall_2017/M1753_OffscreenCanvas]]
*[[CSC/ECE 517 Fall 2017/M1754_Mutation_Testing]]
*[[CSC/ECE 517 Fall 2017/E1794. Student-generated questions added to rubric]]
*[[CSC/ECE 517 Fall 2017/E17A6. Fix account creation over web to work reasonably]]
*[[CSC/ECE 517 Fall 2017/E1792 OSS Visualizations for instructors]]
*[[CSC/ECE 517 Fall 2017/E17A4 Allow calibration to be part of an assignment_Team34]]
*[[CSC/ECE 517 Fall 2017/E17A7_Allow_Reviewers_to_bid_on_what_to_review]]
*[[CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll]]

'''Writing Assignment 2'''
*[[CSC/ECE 517 Fall 2017/E1779. Fix teammate advertisements and requests to join a team]]
*[[CSC/ECE_517_Fall_2017/E1773 Investigate and Fix Expertiza Production Version Runtime Exceptions.rb]]
*[[CSC/ECE_517_Fall_2017/E1774 Metareview fixes and improvements.rb]]
*[[CSC/ECE_517_Fall_2017/E1788 OSS project Maroon Heatmap fixes]]
*[[CSC/ECE 517 Fall 2017/E1781 Topic Management]]
*[[CSC/ECE_517_Fall_2017/E1745_Refactor_response_controller.rb]]
*[[CSC/ECE_517_Fall_2017/E1752 Refactor assignments controller]]
*[[CSC/ECE_517_Fall_2017/E1776_Enhance_Imports]]
*[[CSC/ECE_517_Fall_2017/E1756 TLD Refactor response.rb]]
*[[CSC/ECE_517_Fall_2017/E1772_Refactor reputation_web_service_controller.rb]]
*[[CSC/ECE_517_Fall_2017/M1753_OffscreenCanvas API]]
*[[CSC/ECE_517_Fall_2017/M1754_Mutation Testing on Servo]]
*[[CSC/ECE_517_Fall_2017/E1753 OSS project bidding tests]]
*[[CSC/ECE_517_Fall_2017/E1766_Test team functionality]]
*[[CSC/ECE 517 Fall 2017/E1767 Improve imports]]
*[[CSC/ECE_517_Fall_2017/E1786_OSS project Juniper Bookmark Enhancements]]
*[[CSC/ECE_517_Fall_2017/E1787_OSS project Bronze Score calculations]]
*[[CSC/ECE_517_Fall_2017/E1763 Fix Staggered-Deadline Assignments]]
*[[CSC/ECE 517 Fall 2017/E1757 Introduce a Student View for instructors]]
*[[CSC/ECE 517 Fall 2017/E1769_Refactor assignment_form.rb]]
*[[CSC/ECE 517 Fall 2017/E1754_Feature_test_of_rubric_advice]]
*[[CSC/ECE 517 Fall 2017/E1782 OSS Project Red Assignment Directories]]
*[[CSC/ECE 517 Fall 2017/E1748 Add past-due assignments to task list]]
*[[CSC/ECE 517 Fall 2017/E1761 Test First Development Refactor assignment.rb]]
*[[CSC/ECE_517_Fall_2017/E1780_OSS_Project_Teal_Email_Notification_Enhancements]]
*[[CSC/ECE_517_Fall_2017/E1762 Test various kinds of response-map hierarchies]]
*[[CSC/ECE_517_Fall_2017/E1759 ]]
*[[CSC/ECE_517_Fall_2017/E1749_Test First Development Refactor questionnaire_controller.rb]]
*[[CSC/ECE 517 Fall 2017/E1777 Coherent specification of review requirements.rb]]
*[[CSC/ECE 517 Fall 2017/E1771_Refactor_team.rb]]
*[[CSC/ECE 517 Fall 2017/E1784 Fix mass assignments reported by Brakeman.rb]]

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T22:11:32Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T22:11:25Z

Xli76: /* Files to be Modified */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T22:10:48Z

Xli76: /* Introduction */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T18:59:58Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T18:54:48Z

Xli76: /* Task Description */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T18:45:57Z

Xli76: /* Introduction */

File:Dfgjhdfj333.png

2017-11-07T18:36:53Z

Xli76:

File:Dfssdkfhdsk34.png

2017-11-07T18:12:26Z

Xli76:

File:Fgvv555.png

2017-11-07T18:08:19Z

Xli76:

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T18:02:56Z

Xli76: /* Problem Statement */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T17:32:04Z

Xli76: /* Introduction */

CSC/ECE 517 Fall 2017/E17A9 Lazy loading (infinite scroll) for assignments courses questionnaires and user lists with Jscroll

2017-11-07T16:23:36Z

Xli76: Created page with " == Introduction =="

== Introduction ==

CSC/ECE 517 Fall 2017/E1784 Fix mass assignments reported by Brakeman.rb

2017-11-03T21:39:12Z

Xli76: /* Test */

== About Expertiza ==
Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

== Introduction ==

=== Background ===
Mass Assignment is the name Rails gives to the act of constructing your object with a parameters hash. It is "mass assignment" in that you are assigning multiple values to attributes via a single assignment operator.

The following snippets perform mass assignment of the name and topic attribute of the Post model:

* Post.new(:name => "John", :topic => "Something")

* Post.create(:name => "John", :topic => "Something")

* Post.update_attributes(:name => "John", :topic => "Something")

Without the convenience of mass assignment, we'd have to write an assignment statement for each attribute to achieve the same result. Here's an example:

* attrs = { :name => "John", :topic => "Something" }

* post = Post.new

* post.name = attrs[:name]

* post.topic = attrs[:topic]

Obviously, this can get tedious and painful; so we bow at the feet of laziness and say, yes yes, mass assignment is a good thing.

=== Problem Statement ===
With the help of mass assignment, when we create or update certain object, we do not need to write an assignment statement for each attribute. But mass assignment could cause security vulnerabilities. Hackers could add other parameters to do some bad things. For example:

* @post = Post.new(params[:post])

Typically this is used when the user submits a form rendered by a form_for @post. In an ideal world, the params[:post] hash should only contain the fields we displayed on the form. However, it is trivial easy hackers to pass additional fields in their request, so in effect you're allowing a user to set any fields on @post, not just the ones displayed on the form.

How to Deal With Mass Assignment? In models, we need to add "attr_accessible" to implement the whitelist. For example:

class User < ActiveRecord::Base
attr_accessible :first, :last, :email
end

Here, we're explicitly listing out what can be mass-assigned. Everything else will be disallowed. The advantage here is that if we, say, add an admin flag to the User model, it will automatically be safe from mass-assignment.

For controllers, Rails 4 introduces strong parameters, which is a new approach to protect mass assignment. For example:

class UsersController < ApplicationController
def update
user = User.find(params[:id])
if user.update_attributes(user_params) # see below
redirect_to home_path
else
render :edit
end
end

private

# Require that :user be a key in the params Hash,
# and only accept :first, :last, and :email attributes
def user_params
params.require(:user).permit(:first, :last, :email)
end
end
Now, if you attempt something like user.update_attributes(params), you'll get an error in your application. You must first call permit on the params hash with the keys that are allowed for a specific action.

So our group needs to resolve all these "Unprotected mass assignment" issues according to [https://codeclimate.com/github/expertiza/expertiza/issues?category=security Brakeman report].

== Implementation ==
In models, we need to protect all attributes that we allow to be mass-assigned using "attr_accessible". Previously, we just removed all attributes after "attr_accessible" which fixed "Unprotected mass assignment" but would lead to failure when creating and updating models. So the testes could not pass. Therefore, we added all attributes after "attr_accessible" and tests passed. However, this time, Brakeman would report "Potentially dangerous attribute available for mass assignment". In order to solve "Unprotected mass assignment" and ensure that tests can pass, this is the best we can do.

=== Problem 1: Mass assignment is not restricted using attr_accessible ===
If there was no protection at all in the model file. We added "attr_accessible" and all attributes that we allow to be mass-assigned after it.

[[File:Dfbdfgfdg.png]]

=== Problem 2: Unprotected mass assignment ===
We modified the code in an unobtrusive way, i.e. processing the mass assignment parameters with a function without changing them directly.

==== Parameters come from requests ====
Fix unprotected mass assignment in update

[[File:1ghjvvvvv.png]]

Fix unprotected mass assignment in new

[[File:Ddhghghg.png]]

==== Parameters come from other objects ====
If the parameters come from requests, we can directly use "params.permit()" to implement a whitelist. However, if the parameters come from values of other objects, we cannot directly call that built-in function. Firstly, we need to build a hash with those values and pass the hash into "params". And then we can filter the parameters using permit().

Fix unprotected mass assignment in create

[[File:Assignment.png ]]

Fix unprotected mass assignment in new

[[File:Sdfsvvv.png]]

==== params.permit! ====
[[File:1fdgdfgdfg.png]]

== Test ==
The Expertiza project provides 77 rspec tests under expertiza/spec. After modifying over 140 files, we want to make sure these tests could still pass. It turns out all tests could pass except models/tag_prompt_spec.rb.
[[File:Sjsdkfhsdkf.png]]

In tag_prompt_spec.rb, we will create an instance of Criterion and specify the type of it. However, type is a reserved word for ActiveRecord. When we use type as a reserved word, it specifies the name of a model class.

* (1)If we add :type after attr_accessible in Question(Superclass of Criterion) model

Because you add it after attr_accessible, it will allow you to set the value of type. So, the code in tag_prompt_spec.rb above will try to create an instance of "Criterion" or "Checkbox" or "Text" which should be the subclass of Criterion. However, there are no such subclasses Checkbox and Text. Therefore, the test would fail.

[[File:Dfhkbfdg.png]]

* (2)If we do not add :type after attr_accessible in Question(Superclass of Criterion) model

In this situation, when creating Criterion, the information ("Criterion" or "Checkbox" or "Text") of type will be ignored. All three instances are created with type "Criterion" which is subclass of Criterion(itself). However, in the last test, "expect(tp.html_control(tag_dep, an_long_text)).to eql("")" will check the type these three instances. Because the type of last two instances are actually "Criterion", not "Checkbox" or "Text", the test would fail.
[[File:Sdbfksbgdfg.png]]

== Reference ==
#https://github.com/expertiza/expertiza
#https://codeclimate.com/github/expertiza/expertiza/issues?category=security
#https://apidock.com/rails/ActiveModel/MassAssignmentSecurity/ClassMethods/attr_accessible
#https://docs.google.com/document/d/1rdolBAHxVGI9I0N-cT866AqnfORM2L1_m_bo2gRYRrI/edit#
#https://www.happybearsoftware.com/how-i-avoid-the-rails-mass-assignment-security-mistake
#https://code.tutsplus.com/tutorials/mass-assignment-rails-and-you--net-31695

File:Sdbfksbgdfg.png

2017-11-03T21:28:07Z

Xli76:

File:Sjsdkfhsdkf.png

2017-11-03T21:02:28Z

Xli76:

File:Dfhkbfdg.png

2017-11-03T20:59:03Z

Xli76:

CSC/ECE 517 Fall 2017/E1784 Fix mass assignments reported by Brakeman.rb

2017-11-03T17:30:48Z

Xli76: /* Reference */

== About Expertiza ==
Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

== Introduction ==

=== Background ===
Mass Assignment is the name Rails gives to the act of constructing your object with a parameters hash. It is "mass assignment" in that you are assigning multiple values to attributes via a single assignment operator.

The following snippets perform mass assignment of the name and topic attribute of the Post model:

* Post.new(:name => "John", :topic => "Something")

* Post.create(:name => "John", :topic => "Something")

* Post.update_attributes(:name => "John", :topic => "Something")

Without the convenience of mass assignment, we'd have to write an assignment statement for each attribute to achieve the same result. Here's an example:

* attrs = { :name => "John", :topic => "Something" }

* post = Post.new

* post.name = attrs[:name]

* post.topic = attrs[:topic]

Obviously, this can get tedious and painful; so we bow at the feet of laziness and say, yes yes, mass assignment is a good thing.

=== Problem Statement ===
With the help of mass assignment, when we create or update certain object, we do not need to write an assignment statement for each attribute. But mass assignment could cause security vulnerabilities. Hackers could add other parameters to do some bad things. For example:

* @post = Post.new(params[:post])

Typically this is used when the user submits a form rendered by a form_for @post. In an ideal world, the params[:post] hash should only contain the fields we displayed on the form. However, it is trivial easy hackers to pass additional fields in their request, so in effect you're allowing a user to set any fields on @post, not just the ones displayed on the form.

How to Deal With Mass Assignment? In models, we need to add "attr_accessible" to implement the whitelist. For example:

class User < ActiveRecord::Base
attr_accessible :first, :last, :email
end

Here, we're explicitly listing out what can be mass-assigned. Everything else will be disallowed. The advantage here is that if we, say, add an admin flag to the User model, it will automatically be safe from mass-assignment.

For controllers, Rails 4 introduces strong parameters, which is a new approach to protect mass assignment. For example:

class UsersController < ApplicationController
def update
user = User.find(params[:id])
if user.update_attributes(user_params) # see below
redirect_to home_path
else
render :edit
end
end

private

# Require that :user be a key in the params Hash,
# and only accept :first, :last, and :email attributes
def user_params
params.require(:user).permit(:first, :last, :email)
end
end
Now, if you attempt something like user.update_attributes(params), you'll get an error in your application. You must first call permit on the params hash with the keys that are allowed for a specific action.

So our group needs to resolve all these "Unprotected mass assignment" issues according to [https://codeclimate.com/github/expertiza/expertiza/issues?category=security Brakeman report].

== Implementation ==
In models, we need to protect all attributes that we allow to be mass-assigned using "attr_accessible". Previously, we just removed all attributes after "attr_accessible" which fixed "Unprotected mass assignment" but would lead to failure when creating and updating models. So the testes could not pass. Therefore, we added all attributes after "attr_accessible" and tests passed. However, this time, Brakeman would report "Potentially dangerous attribute available for mass assignment". In order to solve "Unprotected mass assignment" and ensure that tests can pass, this is the best we can do.

=== Problem 1: Mass assignment is not restricted using attr_accessible ===
If there was no protection at all in the model file. We added "attr_accessible" and all attributes that we allow to be mass-assigned after it.

[[File:Dfbdfgfdg.png]]

=== Problem 2: Unprotected mass assignment ===
We modified the code in an unobtrusive way, i.e. processing the mass assignment parameters with a function without changing them directly.

==== Parameters come from requests ====
Fix unprotected mass assignment in update

[[File:1ghjvvvvv.png]]

Fix unprotected mass assignment in new

[[File:Ddhghghg.png]]

==== Parameters come from other objects ====
If the parameters come from requests, we can directly use "params.permit()" to implement a whitelist. However, if the parameters come from values of other objects, we cannot directly call that built-in function. Firstly, we need to build a hash with those values and pass the hash into "params". And then we can filter the parameters using permit().

Fix unprotected mass assignment in create

[[File:Assignment.png ]]

Fix unprotected mass assignment in new

[[File:Sdfsvvv.png]]

==== params.permit! ====
[[File:1fdgdfgdfg.png]]

== Test ==
The Expertiza project provides 77 rspec tests under expertiza/spec. After modifying over 140 files, we want to make sure these tests could still pass.
== Reference ==
#https://github.com/expertiza/expertiza
#https://codeclimate.com/github/expertiza/expertiza/issues?category=security
#https://apidock.com/rails/ActiveModel/MassAssignmentSecurity/ClassMethods/attr_accessible
#https://docs.google.com/document/d/1rdolBAHxVGI9I0N-cT866AqnfORM2L1_m_bo2gRYRrI/edit#
#https://www.happybearsoftware.com/how-i-avoid-the-rails-mass-assignment-security-mistake
#https://code.tutsplus.com/tutorials/mass-assignment-rails-and-you--net-31695

CSC/ECE 517 Fall 2017/E1784 Fix mass assignments reported by Brakeman.rb

2017-11-03T17:06:29Z

Xli76: /* Implementation */

== About Expertiza ==
Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

== Introduction ==

=== Background ===
Mass Assignment is the name Rails gives to the act of constructing your object with a parameters hash. It is "mass assignment" in that you are assigning multiple values to attributes via a single assignment operator.

The following snippets perform mass assignment of the name and topic attribute of the Post model:

* Post.new(:name => "John", :topic => "Something")

* Post.create(:name => "John", :topic => "Something")

* Post.update_attributes(:name => "John", :topic => "Something")

Without the convenience of mass assignment, we'd have to write an assignment statement for each attribute to achieve the same result. Here's an example:

* attrs = { :name => "John", :topic => "Something" }

* post = Post.new

* post.name = attrs[:name]

* post.topic = attrs[:topic]

Obviously, this can get tedious and painful; so we bow at the feet of laziness and say, yes yes, mass assignment is a good thing.

=== Problem Statement ===
With the help of mass assignment, when we create or update certain object, we do not need to write an assignment statement for each attribute. But mass assignment could cause security vulnerabilities. Hackers could add other parameters to do some bad things. For example:

* @post = Post.new(params[:post])

Typically this is used when the user submits a form rendered by a form_for @post. In an ideal world, the params[:post] hash should only contain the fields we displayed on the form. However, it is trivial easy hackers to pass additional fields in their request, so in effect you're allowing a user to set any fields on @post, not just the ones displayed on the form.

How to Deal With Mass Assignment? In models, we need to add "attr_accessible" to implement the whitelist. For example:

class User < ActiveRecord::Base
attr_accessible :first, :last, :email
end

Here, we're explicitly listing out what can be mass-assigned. Everything else will be disallowed. The advantage here is that if we, say, add an admin flag to the User model, it will automatically be safe from mass-assignment.

For controllers, Rails 4 introduces strong parameters, which is a new approach to protect mass assignment. For example:

class UsersController < ApplicationController
def update
user = User.find(params[:id])
if user.update_attributes(user_params) # see below
redirect_to home_path
else
render :edit
end
end

private

# Require that :user be a key in the params Hash,
# and only accept :first, :last, and :email attributes
def user_params
params.require(:user).permit(:first, :last, :email)
end
end
Now, if you attempt something like user.update_attributes(params), you'll get an error in your application. You must first call permit on the params hash with the keys that are allowed for a specific action.

So our group needs to resolve all these "Unprotected mass assignment" issues according to [https://codeclimate.com/github/expertiza/expertiza/issues?category=security Brakeman report].

== Implementation ==
In models, we need to protect all attributes that we allow to be mass-assigned using "attr_accessible". Previously, we just removed all attributes after "attr_accessible" which fixed "Unprotected mass assignment" but would lead to failure when creating and updating models. So the testes could not pass. Therefore, we added all attributes after "attr_accessible" and tests passed. However, this time, Brakeman would report "Potentially dangerous attribute available for mass assignment". In order to solve "Unprotected mass assignment" and ensure that tests can pass, this is the best we can do.

=== Problem 1: Mass assignment is not restricted using attr_accessible ===
If there was no protection at all in the model file. We added "attr_accessible" and all attributes that we allow to be mass-assigned after it.

[[File:Dfbdfgfdg.png]]

=== Problem 2: Unprotected mass assignment ===
We modified the code in an unobtrusive way, i.e. processing the mass assignment parameters with a function without changing them directly.

==== Parameters come from requests ====
Fix unprotected mass assignment in update

[[File:1ghjvvvvv.png]]

Fix unprotected mass assignment in new

[[File:Ddhghghg.png]]

==== Parameters come from other objects ====
If the parameters come from requests, we can directly use "params.permit()" to implement a whitelist. However, if the parameters come from values of other objects, we cannot directly call that built-in function. Firstly, we need to build a hash with those values and pass the hash into "params". And then we can filter the parameters using permit().

Fix unprotected mass assignment in create

[[File:Assignment.png ]]

Fix unprotected mass assignment in new

[[File:Sdfsvvv.png]]

==== params.permit! ====
[[File:1fdgdfgdfg.png]]

== Reference ==
#https://github.com/expertiza/expertiza
#https://codeclimate.com/github/expertiza/expertiza/issues?category=security
#https://apidock.com/rails/ActiveModel/MassAssignmentSecurity/ClassMethods/attr_accessible
#https://docs.google.com/document/d/1rdolBAHxVGI9I0N-cT866AqnfORM2L1_m_bo2gRYRrI/edit#
#https://www.happybearsoftware.com/how-i-avoid-the-rails-mass-assignment-security-mistake
#https://code.tutsplus.com/tutorials/mass-assignment-rails-and-you--net-31695

CSC/ECE 517 Fall 2017/E1784 Fix mass assignments reported by Brakeman.rb

2017-11-03T17:06:13Z

Xli76: /* Implementation */

== About Expertiza ==
Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

== Introduction ==

=== Background ===
Mass Assignment is the name Rails gives to the act of constructing your object with a parameters hash. It is "mass assignment" in that you are assigning multiple values to attributes via a single assignment operator.

The following snippets perform mass assignment of the name and topic attribute of the Post model:

* Post.new(:name => "John", :topic => "Something")

* Post.create(:name => "John", :topic => "Something")

* Post.update_attributes(:name => "John", :topic => "Something")

Without the convenience of mass assignment, we'd have to write an assignment statement for each attribute to achieve the same result. Here's an example:

* attrs = { :name => "John", :topic => "Something" }

* post = Post.new

* post.name = attrs[:name]

* post.topic = attrs[:topic]

Obviously, this can get tedious and painful; so we bow at the feet of laziness and say, yes yes, mass assignment is a good thing.

=== Problem Statement ===
With the help of mass assignment, when we create or update certain object, we do not need to write an assignment statement for each attribute. But mass assignment could cause security vulnerabilities. Hackers could add other parameters to do some bad things. For example:

* @post = Post.new(params[:post])

Typically this is used when the user submits a form rendered by a form_for @post. In an ideal world, the params[:post] hash should only contain the fields we displayed on the form. However, it is trivial easy hackers to pass additional fields in their request, so in effect you're allowing a user to set any fields on @post, not just the ones displayed on the form.

How to Deal With Mass Assignment? In models, we need to add "attr_accessible" to implement the whitelist. For example:

class User < ActiveRecord::Base
attr_accessible :first, :last, :email
end

Here, we're explicitly listing out what can be mass-assigned. Everything else will be disallowed. The advantage here is that if we, say, add an admin flag to the User model, it will automatically be safe from mass-assignment.

For controllers, Rails 4 introduces strong parameters, which is a new approach to protect mass assignment. For example:

class UsersController < ApplicationController
def update
user = User.find(params[:id])
if user.update_attributes(user_params) # see below
redirect_to home_path
else
render :edit
end
end

private

# Require that :user be a key in the params Hash,
# and only accept :first, :last, and :email attributes
def user_params
params.require(:user).permit(:first, :last, :email)
end
end
Now, if you attempt something like user.update_attributes(params), you'll get an error in your application. You must first call permit on the params hash with the keys that are allowed for a specific action.

So our group needs to resolve all these "Unprotected mass assignment" issues according to [https://codeclimate.com/github/expertiza/expertiza/issues?category=security Brakeman report].

== Implementation ==
In models, we need to protect all attributes that we allow to be mass-assigned using "attr_accessible". Previously, we just removed all attributes after "attr_accessible" which fixed "Unprotected mass assignment" but would lead to failure when creating and updating models. So the testes could not pass. Therefore, we added all attributes after "attr_accessible" and tests passed. However, this time, Brakeman would report "Potentially dangerous attribute available for mass assignment". In order to solve "Unprotected mass assignment" and ensure that tests can pass, this is the best we can do.

=== Problem 1: Mass assignment is not restricted using attr_accessible ===
If there was no protection at all in the model file. We added "attr_accessible" and all attributes that we allow to be mass-assigned after it.

[[File:Dfbdfgfdg.png]]

=== Problem 3: Unprotected mass assignment ===
We modified the code in an unobtrusive way, i.e. processing the mass assignment parameters with a function without changing them directly.

==== Parameters come from requests ====
Fix unprotected mass assignment in update

[[File:1ghjvvvvv.png]]

Fix unprotected mass assignment in new

[[File:Ddhghghg.png]]

==== Parameters come from other objects ====
If the parameters come from requests, we can directly use "params.permit()" to implement a whitelist. However, if the parameters come from values of other objects, we cannot directly call that built-in function. Firstly, we need to build a hash with those values and pass the hash into "params". And then we can filter the parameters using permit().

Fix unprotected mass assignment in create

[[File:Assignment.png ]]

Fix unprotected mass assignment in new

[[File:Sdfsvvv.png]]

==== params.permit! ====
[[File:1fdgdfgdfg.png]]

== Reference ==
#https://github.com/expertiza/expertiza
#https://codeclimate.com/github/expertiza/expertiza/issues?category=security
#https://apidock.com/rails/ActiveModel/MassAssignmentSecurity/ClassMethods/attr_accessible
#https://docs.google.com/document/d/1rdolBAHxVGI9I0N-cT866AqnfORM2L1_m_bo2gRYRrI/edit#
#https://www.happybearsoftware.com/how-i-avoid-the-rails-mass-assignment-security-mistake
#https://code.tutsplus.com/tutorials/mass-assignment-rails-and-you--net-31695

File:Dfbdfgfdg.png

2017-11-03T17:05:55Z

Xli76:

CSC/ECE 517 Fall 2017/E1784 Fix mass assignments reported by Brakeman.rb

2017-11-03T17:04:02Z

Xli76: /* Implementation */

== About Expertiza ==
Expertiza is an open source project based on Ruby on Rails framework. Expertiza allows the instructor to create new assignments and customize new or existing assignments. It also allows the instructor to create a list of topics the students can sign up for. Students can form teams in Expertiza to work on various projects and assignments. Students can also peer review other students' submissions. Expertiza supports submission across various document types, including the URLs and wiki pages.

== Introduction ==

=== Background ===
Mass Assignment is the name Rails gives to the act of constructing your object with a parameters hash. It is "mass assignment" in that you are assigning multiple values to attributes via a single assignment operator.

The following snippets perform mass assignment of the name and topic attribute of the Post model:

* Post.new(:name => "John", :topic => "Something")

* Post.create(:name => "John", :topic => "Something")

* Post.update_attributes(:name => "John", :topic => "Something")

Without the convenience of mass assignment, we'd have to write an assignment statement for each attribute to achieve the same result. Here's an example:

* attrs = { :name => "John", :topic => "Something" }

* post = Post.new

* post.name = attrs[:name]

* post.topic = attrs[:topic]

Obviously, this can get tedious and painful; so we bow at the feet of laziness and say, yes yes, mass assignment is a good thing.

=== Problem Statement ===
With the help of mass assignment, when we create or update certain object, we do not need to write an assignment statement for each attribute. But mass assignment could cause security vulnerabilities. Hackers could add other parameters to do some bad things. For example:

* @post = Post.new(params[:post])

Typically this is used when the user submits a form rendered by a form_for @post. In an ideal world, the params[:post] hash should only contain the fields we displayed on the form. However, it is trivial easy hackers to pass additional fields in their request, so in effect you're allowing a user to set any fields on @post, not just the ones displayed on the form.

How to Deal With Mass Assignment? In models, we need to add "attr_accessible" to implement the whitelist. For example:

class User < ActiveRecord::Base
attr_accessible :first, :last, :email
end

Here, we're explicitly listing out what can be mass-assigned. Everything else will be disallowed. The advantage here is that if we, say, add an admin flag to the User model, it will automatically be safe from mass-assignment.

For controllers, Rails 4 introduces strong parameters, which is a new approach to protect mass assignment. For example:

class UsersController < ApplicationController
def update
user = User.find(params[:id])
if user.update_attributes(user_params) # see below
redirect_to home_path
else
render :edit
end
end

private

# Require that :user be a key in the params Hash,
# and only accept :first, :last, and :email attributes
def user_params
params.require(:user).permit(:first, :last, :email)
end
end
Now, if you attempt something like user.update_attributes(params), you'll get an error in your application. You must first call permit on the params hash with the keys that are allowed for a specific action.

So our group needs to resolve all these "Unprotected mass assignment" issues according to [https://codeclimate.com/github/expertiza/expertiza/issues?category=security Brakeman report].

== Implementation ==
In models, we need to protect all attributes that we allow to be mass-assigned using "attr_accessible". Previously, we just removed all attributes after "attr_accessible" which fixed "Unprotected mass assignment" but would lead to failure when creating and updating models. So the testes could not pass. Therefore, we added all attributes after "attr_accessible" and tests passed. However, this time, Brakeman would report "Potentially dangerous attribute available for mass assignment". In order to solve "Unprotected mass assignment" and ensure that tests can pass, this is the best we can do.

=== Problem 1: Mass assignment is not restricted using attr_accessible ===
If there was no protection at all in the model file. We added "attr_accessible" and all attributes that we allow to be mass-assigned after it.

[[File:Anfjdskb23.png]]

=== Problem 3: Unprotected mass assignment ===
We modified the code in an unobtrusive way, i.e. processing the mass assignment parameters with a function without changing them directly.

==== Parameters come from requests ====
Fix unprotected mass assignment in update

[[File:1ghjvvvvv.png]]

Fix unprotected mass assignment in new

[[File:Ddhghghg.png]]

==== Parameters come from other objects ====
If the parameters come from requests, we can directly use "params.permit()" to implement a whitelist. However, if the parameters come from values of other objects, we cannot directly call that built-in function. Firstly, we need to build a hash with those values and pass the hash into "params". And then we can filter the parameters using permit().

Fix unprotected mass assignment in create

[[File:Assignment.png ]]

Fix unprotected mass assignment in new

[[File:Sdfsvvv.png]]

==== params.permit! ====
[[File:1fdgdfgdfg.png]]

== Reference ==
#https://github.com/expertiza/expertiza
#https://codeclimate.com/github/expertiza/expertiza/issues?category=security
#https://apidock.com/rails/ActiveModel/MassAssignmentSecurity/ClassMethods/attr_accessible
#https://docs.google.com/document/d/1rdolBAHxVGI9I0N-cT866AqnfORM2L1_m_bo2gRYRrI/edit#
#https://www.happybearsoftware.com/how-i-avoid-the-rails-mass-assignment-security-mistake
#https://code.tutsplus.com/tutorials/mass-assignment-rails-and-you--net-31695