Expertiza_Wiki - User contributions [en]

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-12-04T20:54:59Z

Krkulkar: /* Links */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

=== Solution ===
*'''users/list'''
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

Code added to app/controllers/users_controller.rb
def list
user = session[:user]
@users=Kaminari.paginate_array(user.get_user_list).page(params[:page])
end

Code added to app/models/user.rb
<%= paginate @user %>

Code added to app/views/users/list.html.erb
<%= paginate @users %>

*'''grades/view'''
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

'''Issues faced during implementation of this solution''':-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.We also created a route for this controller method.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

=='''Test Plan'''==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup), this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
Username :- instructor6
Password :- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

=== Links ===
* Project Demo : https://www.youtube.com/watch?v=qQi6yqbmST4&t=4s
* Project Repository: https://github.com/sahildorwat/expertiza
* Pull request: https://github.com/expertiza/expertiza/pull/1122
* Github issue: https://github.com/expertiza/expertiza/issues/1121

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-12-04T20:40:12Z

Krkulkar: /* Links */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

=== Solution ===
*'''users/list'''
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

Code added to app/controllers/users_controller.rb
def list
user = session[:user]
@users=Kaminari.paginate_array(user.get_user_list).page(params[:page])
end

Code added to app/models/user.rb
<%= paginate @user %>

Code added to app/views/users/list.html.erb
<%= paginate @users %>

*'''grades/view'''
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

'''Issues faced during implementation of this solution''':-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.We also created a route for this controller method.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

=='''Test Plan'''==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup), this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
Username :- instructor6
Password :- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

=== Links ===
* Project Demo : https://www.youtube.com/watch?v=qQi6yqbmST4&t=4s
* Project Repository: https://github.com/sahildorwat/expertiza
* Pull request: https://github.com/expertiza/expertiza/pull/1120
* Github issue: https://github.com/expertiza/expertiza/issues/1121

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-12-04T18:59:13Z

Krkulkar: /* Links */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T23:01:06Z

Krkulkar: /* Solution */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T23:00:42Z

Krkulkar: /* Solution */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

=== Solution ===
*'''users/list'''
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

Code added to app/controllers/users_controller.rb
def list
user = session[:user]
@users=Kaminari.paginate_array(user.get_user_list).page(params[:page])
end

Code added to app/models/user.rb
<%= paginate @user %>

Code added to app/views/users/list.html.erb
<%= paginate @users %>

*'''grades/view'''
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.We also created a route for this controller method.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

=='''Test Plan'''==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup), this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
Username :- instructor6
Password :- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:57:04Z

Krkulkar: /* Initial Setup for testing */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

=='''Test Plan'''==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup), this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
Username :- instructor6
Password :- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:56:51Z

Krkulkar: /* Initial Setup for testing */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

=='''Test Plan'''==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup), this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:56:37Z

Krkulkar: /* Test Plan */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

=='''Test Plan'''==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:56:11Z

Krkulkar: /* Analysis of the page expertiza.ncsu.edu/tree_display/list */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the pages''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:55:54Z

Krkulkar: /* Identification of pages */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

== '''Analysis of the page expertiza.ncsu.edu/tree_display/list''' ==

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:54:46Z

Krkulkar: /* Identification of pages */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

=='''Identification of pages''' ==
===Analysis of the page expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:54:19Z

Krkulkar: /* Approach */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

=='''Approach'''==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:54:10Z

Krkulkar: /* Project Design */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== '''Project Design''' ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:53:58Z

Krkulkar: /* Pages which needs to be optimized */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== '''Pages which needs to be optimized''' ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:53:47Z

Krkulkar: /* Gems Installed */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== '''Gems Installed''' ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:53:38Z

Krkulkar: /* Project Description */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== '''Project Description''' ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:53:27Z

Krkulkar: /* Introduction */

= Optimization for page loading in Expertiza =

== '''Introduction''' ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:52:44Z

Krkulkar: /* Testing */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:51:38Z

Krkulkar: /* Testing */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull. You will notice a substantial reduction in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:49:10Z

Krkulkar: /* Test Plan */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.
The existing code consists of a for loop in views/grades/_teams.html.erb, which renders the partial "views/grades/_tabbing.html.erb" which displays the data repeatedly. Instead, this partial could be rendered through AJAX or JQuery, whenever the expand button is clicked. This would save redundant loading of data and thus reduce the loading time of the page.

Issues faced during implementation of this solution:-

*While implementing this fix, we first created a controller method, which would render the required partial "views/grades/_tabbing.html.erb" when called.
*We passed the required data through the Rails params hash. However, since one argument that needed to be passed (tscore) is a Hash, when it enters the params hash, it is getting converted to a String.
*We tried passing the hash as it is, but after doing this all the data in the hash is not retained, i.e. there is a loss of data.
*We also tried converting the String back into Hash by using the eval method, and to_unsafe_h method.
*However, these methods require the hash to be present in a normal form i.e. key value pairs.
*But our hash has values as Objects, thus these methods cannot be applied
*Thus, we are unable to access the data in the String-hash.
*If this could be done, then our approach would result in successfully decreasing the loading time of the webpage.

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:- password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:47:14Z

Krkulkar: /* Approach */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- The approach to reduce loading time on this page was by implementing pagination. Instead of displaying all users and their data all at once, we display a certain number of users on one page, and rest of the users can be viewed by accessing the later pages. This reduces loading time of the page by reducing the amount of data to be fetched.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.

Issues faced during implementation of this solution:-
1)

2)

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:44:23Z

Krkulkar: /* Gems Installed */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development.
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof :- A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :- fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari :- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- Pagination was used to optimze. This was done using the Kaminari gem for Rails.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.

Issues faced during implementation of this solution:-
1)

2)

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:43:48Z

Krkulkar: /* Pages which needs to be optimized */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari:- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view?id=<ID of the team>

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- Pagination was used to optimze. This was done using the Kaminari gem for Rails.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.

Issues faced during implementation of this solution:-
1)

2)

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:43:00Z

Krkulkar: /* PROJECT DESIGN */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari:- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view

2) expertiza.ncsu.edu/users/list

== Project Design ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- Pagination was used to optimze. This was done using the Kaminari gem for Rails.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.

Issues faced during implementation of this solution:-
1)

2)

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:42:44Z

Krkulkar: /* PROJECT DESIGN */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari:- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view

2) expertiza.ncsu.edu/users/list

== PROJECT DESIGN ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern - The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle - We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization - In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- Pagination was used to optimze. This was done using the Kaminari gem for Rails.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.

Issues faced during implementation of this solution:-
1)

2)

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-29T22:41:46Z

Krkulkar: /* Approach */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.
*Kaminari:- A Scope & Engine based, clean, powerful, customizable and sophisticated paginator for modern web app frameworks and ORMs

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view

2) expertiza.ncsu.edu/users/list

== PROJECT DESIGN ==

For the purpose of this project, experts in the expertiza domain are Instructors.

The following two patterns are implemented in the project -

1.MVC Pattern – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).

2.DRY Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.

3.Optimization-In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be implemented in order to optimize the webpage rendering.

Optimization approach for the pages:
*users/list :- Pagination was used to optimze. This was done using the Kaminari gem for Rails.
*grades/view : Ajax must be used for the optimization. Currently, all the data for all the teams present is loaded when the webpage is accessed. Instead, data for particular teams can be loaded when the user clicks on the expand button for that particular team. Thus, data for each team should be loaded asynchronously with ajax.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-

[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Solution ==
*users/list
The time taken to load the page was high because the server had to access through hundreds of rows in the database .The page was rendering the entire row of database at the same time. Pagination helped to split the database into manageable chunks of data.

*grades/view
The grades/view is the page where the instructor accesses the grades statistics of each student.
The reason for the slow down in the page is that the data for all the teams were loaded as soon as the page was loaded.The design of the page has an individual tab for each team, hence loading the entire data as soon as the page is loaded is quite redundant as the instructor needs to access the scores of each individual team at a given time.
The solution to this problem is to access the individual team statistics from the database when the instructor clicks the tab specified for the particular individual.

Issues faced during implementation of this solution:-
1)

2)

==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

===Automated Test===

There can be no automated test for this program since the improvement in the performance is due to the inclusion of the pagination gem.

Hence testing if the pagination gem will work is equivalent to checking if rails is working.

As testing rails is not recommended no automated test is added for changes made.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-27T01:47:47Z

Krkulkar: /* Approach */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view

2) expertiza.ncsu.edu/users/list

== PROJECT DESIGN ==

For the purpose of this project, experts in the expertiza domain are Instructors.
The following two patterns are implemented in the project -
1.MVC Pattern – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).
2.DRY Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.
3.Optimization-In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task, we identified the factors which are causing the lag. This was done by using flamegraph generation and rack-mini-profiler statistics. The statistics generated by rack miniprofiler helped us to understand the time taken by each component to get loaded. From these statistics, we identified the model/controller/view/database query which is causing this latency. From here, we identified the methods which needed to be refactored in order to optimize the webpage rendering.

To optimize the page users/list, we implemented pagination. This was done using the Kaminari gem for Rails.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-
[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Proposed Solution ==
The time taken to load the page is high due to the time taken for the action children_node_ng. The time taken to execute it is 10564.8 milliseconds. This is due to the 1123 SQL calls which are performed in the given action. This delay can be reduced by reducing the total number of SQL calls, which can be reduced by identifying the methods causing this delay as well as by refactoring the code. The solution is to refactor the code which has been attached below to improve the time taken to perform this action.

def update_fnode_children(fnode, tmp_res)
# fnode is short for foldernode which is the parent node
# ch_nodes are childrens
# cnode = fnode.get_children("created_at", "desc", 2, nil, nil)
ch_nodes = fnode.get_children(nil, nil, session[:user].id, nil, nil)
tmp_res[fnode.get_name] = ch_nodes
end

# initialize parent node and update child nodes for it
def initialize_fnode_update_children(params, node, tmp_res)
fnode = (params[:reactParams][:nodeType]).constantize.new
node.each do |a|
fnode[a[0]] = a[1]
end
update_fnode_children(fnode, tmp_res)
end

# for child nodes
def children_node_ng
child_nodes = child_nodes_from_params(params[:reactParams][:child_nodes])
tmp_res = {}
child_nodes.each do |node|
initialize_fnode_update_children(params, node, tmp_res)
end
res = res_node_for_child(tmp_res)
respond_to do |format|
format.html { render json: res }
end
end
==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

3)Check the time taken to load the page localhost:3000/grades/view on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

4)Check the time taken to load the page localhost:3000/users/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-27T01:43:07Z

Krkulkar: /* Pages which needs to be optimized */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.

== Pages which needs to be optimized ==

1) expertiza.ncsu.edu/grades/view

2) expertiza.ncsu.edu/users/list

== PROJECT DESIGN ==

For the purpose of this project, experts in the expertiza domain are Instructors.
The following two patterns are implemented in the project -
1.MVC Pattern – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).
2.DRY Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.
3.Optimization-In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task,we need to identify the elements which are causing this lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics. The statistics generated by rack miniprofiler help us to understand the time taken by each component to get loaded. From these statistics, we can identify the model/controller/view/database query which is causing this latency. From here, we can identify the methods which need to be refactored in order to optimize the webpage rendering.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-
[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Proposed Solution ==
The time taken to load the page is high due to the time taken for the action children_node_ng. The time taken to execute it is 10564.8 milliseconds. This is due to the 1123 SQL calls which are performed in the given action. This delay can be reduced by reducing the total number of SQL calls, which can be reduced by identifying the methods causing this delay as well as by refactoring the code. The solution is to refactor the code which has been attached below to improve the time taken to perform this action.

def update_fnode_children(fnode, tmp_res)
# fnode is short for foldernode which is the parent node
# ch_nodes are childrens
# cnode = fnode.get_children("created_at", "desc", 2, nil, nil)
ch_nodes = fnode.get_children(nil, nil, session[:user].id, nil, nil)
tmp_res[fnode.get_name] = ch_nodes
end

# initialize parent node and update child nodes for it
def initialize_fnode_update_children(params, node, tmp_res)
fnode = (params[:reactParams][:nodeType]).constantize.new
node.each do |a|
fnode[a[0]] = a[1]
end
update_fnode_children(fnode, tmp_res)
end

# for child nodes
def children_node_ng
child_nodes = child_nodes_from_params(params[:reactParams][:child_nodes])
tmp_res = {}
child_nodes.each do |node|
initialize_fnode_update_children(params, node, tmp_res)
end
res = res_node_for_child(tmp_res)
respond_to do |format|
format.html { render json: res }
end
end
==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

3)Check the time taken to load the page localhost:3000/grades/view on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

4)Check the time taken to load the page localhost:3000/users/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-27T01:42:54Z

Krkulkar: /* Pages which needs to be optimized */

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Project Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.

== Pages which needs to be optimized ==

1)expertiza.ncsu.edu/grades/view

2)expertiza.ncsu.edu/users/list

== PROJECT DESIGN ==

For the purpose of this project, experts in the expertiza domain are Instructors.
The following two patterns are implemented in the project -
1.MVC Pattern – The project is implemented in Ruby on Rails that uses MVC architecture. It separates an application’s data model, user interface, and control logic into three distinct components (model, view, and controller, respectively).
2.DRY Principle – We are trying to reuse the existing functionalities in Expertiza, thus avoiding code duplication. Whenever possible, code modification based on the existing classes, controllers, or tables will be done instead of creating the new one.
3.Optimization-In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics.

==Approach==
To complete the given task,we need to identify the elements which are causing this lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics. The statistics generated by rack miniprofiler help us to understand the time taken by each component to get loaded. From these statistics, we can identify the model/controller/view/database query which is causing this latency. From here, we can identify the methods which need to be refactored in order to optimize the webpage rendering.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-
[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===

The following is the rack-mini-profiler statistics for the page:-

[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

== Proposed Solution ==
The time taken to load the page is high due to the time taken for the action children_node_ng. The time taken to execute it is 10564.8 milliseconds. This is due to the 1123 SQL calls which are performed in the given action. This delay can be reduced by reducing the total number of SQL calls, which can be reduced by identifying the methods causing this delay as well as by refactoring the code. The solution is to refactor the code which has been attached below to improve the time taken to perform this action.

def update_fnode_children(fnode, tmp_res)
# fnode is short for foldernode which is the parent node
# ch_nodes are childrens
# cnode = fnode.get_children("created_at", "desc", 2, nil, nil)
ch_nodes = fnode.get_children(nil, nil, session[:user].id, nil, nil)
tmp_res[fnode.get_name] = ch_nodes
end

# initialize parent node and update child nodes for it
def initialize_fnode_update_children(params, node, tmp_res)
fnode = (params[:reactParams][:nodeType]).constantize.new
node.each do |a|
fnode[a[0]] = a[1]
end
update_fnode_children(fnode, tmp_res)
end

# for child nodes
def children_node_ng
child_nodes = child_nodes_from_params(params[:reactParams][:child_nodes])
tmp_res = {}
child_nodes.each do |node|
initialize_fnode_update_children(params, node, tmp_res)
end
res = res_node_for_child(tmp_res)
respond_to do |format|
format.html { render json: res }
end
end
==Test Plan==
The project is based on optimizing the time required to load the pages which has been mentioned above,in order to test that there is an improvement in the time taken to load these pages one will have to compare the initial loading time and the time taken after the code has been modified.

===Initial Setup for testing===

1)Clone the git repository.

2)Go to the commit on 3rd November(name: initial setup),this is the code which is originally provided by the professor and has not been modified. It however consists of the two gems required to check the loading time.

3)To login as instructor :-
USERNAME :- instructor6
Password:-password

===Testing===

1)Check the time taken to load the page localhost:3000/tree_display/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

2)Check the time taken to load the page localhost:3000/review_mapping/response_report on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

3)Check the time taken to load the page localhost:3000/grades/view on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

4)Check the time taken to load the page localhost:3000/users/list on the left corner and compare the same for the latest pull,a substantial difference in the loading time.

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:11:51Z

Krkulkar: /* MiniProfiler Statistics */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:11:07Z

Krkulkar: /* MiniProfiler Statistics */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:09:23Z

Krkulkar: /* Proposed Solution */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:07:34Z

Krkulkar: /* Solution */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:07:23Z

Krkulkar:

= Optimization for page loading in Expertiza =

== Introduction ==
Expertiza is an online system that is used by students to view/submit assignments and review others' work. Expertiza also provides tools to visualize the scores and gauge the improvements made during the course semester. It also facilitates and monitors team projects. It is targeted at educational and non-profit organizations. The project is funded by the National Software Foundation (NSF), NCSU Learning in a Technology-Rich Environment (LITRE) program, the NCSU Faculty Center for Teaching and Learning, the NCSU STEM Initiative, and the Center for Advanced Computing and Communication.

Expertiza is an open-source project with the source code available as a public repository on GitHub. It is developed using Ruby on Rails and is increasingly becoming robust thanks to the innumerable bugs being fixed by the community. The project has a micro-blog on SourceForge where the developer community report bugs and document updates.

== Task Description ==

1) Identifying the Expertiza pages which take time to load using Rack-mini-profiler and Flamegraphs.

2) Propose fixes which would improve the Expertiza project.

3) Optimizing the view, models and controllers for few of these corresponding fixes to improve the load time of these pages.

== Gems Installed ==

*rack-mini-profiler :- Middleware that displays speed badge for every html page. Designed to work both in production and in development
*flamegraphs :- Flame graphs are a visualization of profiled software, allowing the most frequent code-paths to be identified quickly and accurately.
*stackprof:-A sampling call-stack profiler for ruby 2.1+.Downloaded as a dependency for rack-mini-profiler.
*fast_stack :-fast_stack is dynamically resizable data structure optimized for fast iteration over the large arrays of similar elements avoiding memory fragmentation.

== Pages which needs to be optimized ==

1)expertiza.ncsu.edu/tree_display/list

2)expertiza.ncsu.edu/review_mapping/response_report

3)expertiza.ncsu.edu/grades/view

4)expertiza.ncsu.edu/users/list

== Approach ==
In order to optimize, we need to understand the factors which are causing the lag. This can be identified by using the flamegraph generation method and rack-mini-profiler statistics. The statistics generated by rack miniprofiler help us to understand the time taken by each component to get loaded. From these statistics, we can identify the model/controller/view/database query which is causing this latency. From here, we can identify the methods which need to be refactored in order to optimize the webpage rendering.

==Identification of pages ==
===Analysis of page 1: expertiza.ncsu.edu/tree_display/list ===

===FlameGraph Statistics===
The following is the flamegraph for this page:-
[[File:f.png]]

The X axis of the FlameGraph represents the time taken to load the page. It can give a clear picture of where the expertiza page is getting bogged down. The widest layers take the longest to run. They’re the areas one should look into, because speeding them up could have the biggest impact. As we can see over here, most of the time taken over here is in the controller action.

===MiniProfiler Statistics===
The following is the rack-mini-profiler statistics for the page:-
[[File:tre1.png]]

The box which is seen in the leftmost corner is the MiniProfiler statistics. MiniProfiler gives a constant notification of how long each page takes to load. The miniprofiler statistics show the time taken to render the view as well as the corresponding SQL calls.

[[File:tree2.png]]

This image shows the time and the number of SQL calls which are required for each rendering.

[[File:tree5.png]]

Here as we can see the total number of SQL calls required for children_node_ng is 1123. This is causing the latency in the entire page by 10564.8 ms. These statistics helped us to identify the method which needs to be refactored.

[[File:tree6.png]]

The total number of SQL calls required for 2 components here is 2 each. Hence we do not need to refactor this component.

===Identified Component causing the delay===
tree_display_controller

The following methods are consuming most of the time in this code :-

1)update_fnode_children

2)initialize_fnode_update_children

3)children_node_ng

These methods are from line 172-201 in this controller.

===Solution===
So here as we can see the time taken to load the page is high due to the time taken for the action :children_node_ng.The time taken to execute it is 10564.8 milliseconds.This is due to the 1123 SQL calls which are performed in the given action .This delay can be reduced by reducing the total number of SQL calls which can be basically reduced by identifying the methods causing this delay as well as by refactoring the code. The solution is to refactor the code which has been attached below to improve the time taken to perform this action.

def update_fnode_children(fnode, tmp_res)
# fnode is short for foldernode which is the parent node
# ch_nodes are childrens
# cnode = fnode.get_children("created_at", "desc", 2, nil, nil)
ch_nodes = fnode.get_children(nil, nil, session[:user].id, nil, nil)
tmp_res[fnode.get_name] = ch_nodes
end

# initialize parent node and update child nodes for it
def initialize_fnode_update_children(params, node, tmp_res)
fnode = (params[:reactParams][:nodeType]).constantize.new
node.each do |a|
fnode[a[0]] = a[1]
end
update_fnode_children(fnode, tmp_res)
end

# for child nodes
def children_node_ng
child_nodes = child_nodes_from_params(params[:reactParams][:child_nodes])
tmp_res = {}
child_nodes.each do |node|
initialize_fnode_update_children(params, node, tmp_res)
end
res = res_node_for_child(tmp_res)
respond_to do |format|
format.html { render json: res }
end
end

== Proposed Solution ==
So here as we can see the time taken to load the page is high due to the time taken for the action :children_node_ng.The time taken to execute it is 10564.8 milliseconds.This is due to the 1123 SQL calls which are performed in the given action .This delay can be reduced by reducing the total number of SQL calls which can be basically reduced by identifying the methods causing this delay as well as by refactoring the code. The solution is to refactor the code which has been attached below to improve the time taken to perform this action.

def update_fnode_children(fnode, tmp_res)
# fnode is short for foldernode which is the parent node
# ch_nodes are childrens
# cnode = fnode.get_children("created_at", "desc", 2, nil, nil)
ch_nodes = fnode.get_children(nil, nil, session[:user].id, nil, nil)
tmp_res[fnode.get_name] = ch_nodes
end

# initialize parent node and update child nodes for it
def initialize_fnode_update_children(params, node, tmp_res)
fnode = (params[:reactParams][:nodeType]).constantize.new
node.each do |a|
fnode[a[0]] = a[1]
end
update_fnode_children(fnode, tmp_res)
end

# for child nodes
def children_node_ng
child_nodes = child_nodes_from_params(params[:reactParams][:child_nodes])
tmp_res = {}
child_nodes.each do |node|
initialize_fnode_update_children(params, node, tmp_res)
end
res = res_node_for_child(tmp_res)
respond_to do |format|
format.html { render json: res }
end
end

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:04:27Z

Krkulkar: /* Task Description */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:03:58Z

Krkulkar: /* MiniProfiler Statistics */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:02:57Z

Krkulkar: /* FlameGraph Statistics */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:02:33Z

Krkulkar: /* FlameGraph Statistics */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:01:44Z

Krkulkar: /* Analysis of page1 */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:01:05Z

Krkulkar: /* Approach */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T22:00:09Z

Krkulkar: /* Approach */

CSC/ECE 517 Fall 2017/E17A8. Use a profiler to identify the problems / pages that take some time to load & fix them

2017-11-06T21:59:15Z

Krkulkar: /* Task Description */

CSC/ECE 517 Fall 2017/E1766 Test team functionality

2017-10-28T21:35:16Z

Krkulkar: /* Functional Tests Implementation */

== Background ==
Expertiza is a peer review based system which provides incremental learning from the class. This project has been developed together by faculty and students using Ruby on Rails framework. Expertiza allows the instructor to create, edit and delete assignments, create new assignment topics, assign them to a particular class or selected students, have students work on teams and then review each other's assignments at the end. For the students, they can signup for topics, form teams, and submit their projects and assignments. Students then review the work done by other students and give suggestions to improve. Teams after reviews are allotted scores and they can refer to the peer comments to further improve their work. It also supports submission of different file types for assignments, including the URLs and wiki pages.

== Purpose ==
The purpose of this task is to write functional tests for team functionality. Once an assignment is out, a student can select this assignment and others can join in the team. To test this functionality we wrote functional tests for the various scenarios.
One such scenario is :

* Once the assignment is out and a student selects it, he/she can send out invites to other students to join the team.
* Invited students can accept the invitation and join the team.

== Functional Tests ==

Functional tests ensure that the functionalities of a software system are working as expected. To write our functional tests, we used the Capybara gem available for Ruby.
Capybara gem allows a user to test their web application through simulations.

== Test Plan (Implementation of functional tests) ==

=== Functional Tests for Create group assignment ===

In Expertiza instructor can create public assignment for particular course, define the required rubrics and add students to that assignment.

The following test spec has been written to create public group assignment

<pre>
def create_new_assignment
login_as("instructor6")
visit "/assignments/new?private=0"

fill_in 'assignment_form_assignment_name', with: 'public assignment for test'
select('Course 2', from: 'assignment_form_assignment_course_id')
fill_in 'assignment_form_assignment_directory_path', with: 'testDirectory'
fill_in 'assignment_form_assignment_spec_location', with: 'testLocation'
check("assignment_form_assignment_microtask")
check("team_assignment")
fill_in 'assignment_form_assignment_max_team_size', with: '3', visible: false
check("assignment_form_assignment_reviews_visible_to_all")
check("assignment_form_assignment_is_calibrated")
check("assignment_form_assignment_availability_flag")
expect(page).to have_select("assignment_form[assignment][reputation_algorithm]", options: ['--', 'Hamer', 'Lauw'])

click_button 'Create'
end
</pre>

Add topics to the assignment

<pre>
def add_topic_to_assignment assignment
visit "/assignments/#{assignment.id}/edit"
click_link 'Topics'
click_link 'New topic'
fill_in 'topic_topic_identifier', with: '112'
fill_in 'topic_topic_name', with: 'test_topic_1'
fill_in 'topic_category', with: 'test_topic_1'
fill_in 'topic_max_choosers', with: 3
click_button 'Create'
create(:assignment_due_date)
create_list(:participant, 3)
end
</pre>

=== Functional Tests for Log in as student and selecting a topic of assignment ===

When student login into his Expertiza account, he can check for new assignment under "Assignments" tab and choose the topic of his interest. Student then can send invitation to other students who are also enrolled in same course.
We have written following test spec to impersonate as a student and select a topic

<pre>
it "should impersonate as student" do
user = User.find_by(name: "student2064")
stub_current_user(user, user.role.name, user.role)
visit '/student_task/list'
# Assignment name
expect(page).to have_content('public assignment for test')
click_link 'public assignment for test'
expect(page).to have_content('Submit or Review work for public assignment for test')

click_link 'Signup sheet'
expect(page).to have_content('Signup sheet for public assignment for test assignment')
assignment_id = Assignment.first.id
visit "/sign_up_sheet/sign_up?id=#{assignment_id}&topic_id=1"
expect(page).to have_content('Your topic(s): test_topic_1')
end
</pre>

Sending invitations to other students
<pre>
visit '/student_task/list'
click_link 'public assignment for test'
click_link 'Your team'
expect(page).to have_content('public assignment for test_Team1')
fill_in 'user_name', with: 'student2065'
click_button 'Invite'
fill_in 'user_name', with: 'student2066'
click_button 'Invite'
expect(page).to have_content('student2065')
</pre>

=== Functional Tests for accepting an invitation ===

In Expertiza assignments, the only way for a student to join an existing team is to be invited by the leader.

Following test spec has been written to accept the invitation to join the team and verify that the assignment topic is assigned to all users

<pre>
it "joins the team" do
user = User.find_by(name: "student2065")
stub_current_user(user, user.role.name, user.role)
visit '/student_task/list'
expect(page).to have_content('public assignment for test')
visit '/invitation/accept?inv_id=1&student_id=1&team_id=1'
visit '/student_teams/view?student_id=1'
expect(page).to have_content('Team Information for public assignment for test')

# to test invalid case - student who is not part of the team does not have created assignment
user = User.find_by(name: "student2066")
stub_current_user(user, user.role.name, user.role)
visit '/student_task/list'
page.should has_no_content?('public assignment for test')
end
</pre>

===Functional Tests for whether students choosing the same topic are in the same team===

Since the only way for students to be in a team is by invitation, students who choose the same topic will be in different team. Our test is to let two different students select the same topic and see if their teammates contains each other. Therefore, first we let ‘student2064’ and 'student2065' sign for the same topic, and in their pages, to see if there is 'student2065' or 'student2064'.

<pre>
expect(page).to have_content "Team Name"
expect(page).to have_content "student2065"
expect(page).to have_content "student2064"
</pre>

== Running the tests ==

The tests can be run on the terminal by navigating to the /spec/features directory using the command:
<pre>
rspec team_functionalities_spec.rb
</pre>

Whether the test fails or succeeds, allows us to determine which parts of the system are functioning properly.

== Result ==

Coverage increased (+0.3%) to 51.108% when pulling 4915488 on ankit13jain:master into 781e456 on expertiza:master.

== External Links ==
# link for forked repository [[https://github.com/ankit13jain/expertiza]]
# Github link for original repository [[https://github.com/expertiza/expertiza.git]]
# Github link for Capybara [[https://github.com/jnicklas/capybara.git]]