Expertiza_Wiki - User contributions [en]

Independent Study Spring 2019/Errbit

2019-05-08T14:39:05Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[ Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made a new Heroku account with the following credentails:
Email: expertiza-support@lists.ncsu.edu
Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

===Deploy Errbit on Heroku===
To deploy Errbit on Heroku, so that we could configure it to catch errors for Expertiza, the following steps were taken:

*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[[File:Build.png|frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png|frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png|frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[[File:ErrorDetls.png|frame|upright|center]]
*Similar errors are grouped together.
[[File:ErrorsGroup.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-08T14:38:10Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[ Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made a new Heroku account with the following credentails:
Email: expertiza-support@lists.ncsu.edu
Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

===Deploy Errbit on Heroku===
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[[File:Build.png|frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png|frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png|frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[[File:ErrorDetls.png|frame|upright|center]]
*Similar errors are grouped together.
[[File:ErrorsGroup.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-08T14:37:41Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[ Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made a new Heroku account with the following credentails:
Email: expertiza-support@lists.ncsu.edu
Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[[File:Build.png|frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png|frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png|frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[[File:ErrorDetls.png|frame|upright|center]]
*Similar errors are grouped together.
[[File:ErrorsGroup.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-08T14:36:54Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[ Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[[File:Build.png|frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png|frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png|frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[[File:ErrorDetls.png|frame|upright|center]]
*Similar errors are grouped together.
[[File:ErrorsGroup.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-08T14:35:58Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[[File:Build.png|frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png|frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png|frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[[File:ErrorDetls.png|frame|upright|center]]
*Similar errors are grouped together.
[[File:ErrorsGroup.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-08T14:32:23Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[[File:Build.png||frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png||frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png||frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[[File:ErrorDetls.png||frame|upright|center]]
*Similar errors are grouped together.
[[File:ErrorsGroup.png||frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-08T14:31:50Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===
*Deployment of Errbit on Heroku, showing successful build.
[File:Build.png||frame|upright|center]]
*Login to Errbit using the admin credentials we specified.
[[File:ErrbitLogin.png||frame|upright|center]]
*Create a new App on Errbit to catch Errors for Expertiza
[[File:ExpertizaApp.png||frame|upright|center]]
*Errors from Expertiza are caught in this manner and displayed.
[File:ErrorDetls.png||frame|upright|center]]
*Similar errors are grouped together.
[File:ErrorsGroup.png||frame|upright|center]]

File:ErrorsGroup.png

2019-05-08T14:25:22Z

Psingh25:

File:ErrorDetls.png

2019-05-08T14:24:43Z

Psingh25:

File:ErrbitLogin.png

2019-05-08T14:23:54Z

Psingh25: uploaded a new version of "File:ErrbitLogin.png"

File:Build.png

2019-05-08T14:23:15Z

Psingh25:

File:ExpertizaApp.png

2019-05-08T14:21:49Z

Psingh25:

Independent Study Spring 2019/Errbit

2019-05-08T14:19:19Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

===Errbit Interface===

File:ErrbitLogin.png

2019-05-08T14:18:01Z

Psingh25:

Independent Study Spring 2019/Errbit

2019-05-08T14:16:17Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==Components==
*Errbit
*Expertiza
*MongoDB

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku and named it errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following to deploy.
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit named Expertiza
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end
On starting the server, the errors would now be caught by Errbit.

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''

*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

You could access the app using:
https://errbit-expertiza2019.herokuapp.com/
UserName: expertiza-support@lists.ncsu.edu
Password: errbit

Independent Study Spring 2019/Errbit

2019-05-08T13:22:29Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''
*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬
*Setup tasks to fetch from upstream and rebase and then push to Heroku.
*Task to migrate the database into production
*To deploy using these tasks, run:
rake deploy:production DEPLOY_BRANCH=production

Independent Study Spring 2019/Errbit

2019-05-08T08:02:26Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''
*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬

Independent Study Spring 2019/Errbit

2019-05-08T07:20:02Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy its configuration and place it in the expertiza/config/initializers/airbrake.rb file.

Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from upstream/master into the forked repository and deploy on Heroku.'''
*Add the original GitHub repository as a "remote" called upstream to [https://github.com/errbit/errbit Errbit]
*Add the empty Git repository made during app creation on Heroku as a "remote" called "production".
In the .git/config, add:

[remote "upstream"]¬
url = https://github.com/errbit/errbit.git¬
fetch = +refs/heads/*:refs/remotes/upstream/*¬
[remote "production"]¬
url = https://git.heroku.com/errbit-expertiza2019.git¬
fetch = +refs/heads/*:refs/remotes/heroku/*¬

Independent Study Spring 2019/Errbit

2019-05-08T07:00:04Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?
"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create an addon.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

To periodically clear resolved errors to free up space, With the cron add-on:Install the heroku cron addon, to clear resolved errors daily:
heroku addons:add cron:daily

Run the following
git add .
git commit -m “new creds”
git push heroku master

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy it's configuration and place it in the expertiza/config/initializers/airbrake.rb file.
Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

Independent Study Spring 2019/Errbit

2019-05-08T06:17:34Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?

"expertiza-support@lists.ncsu.edu"
end
def admin_pass
return 'errbit' if heroku_pr_review_app?
@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create addon one.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

git add . 
git commit -m “new creds” 
git push heroku master 

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy it's configuration and place it in the expertiza/config/initializers/airbrake.rb file.
Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

Independent Study Spring 2019/Errbit

2019-05-08T06:16:17Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run "rake db:seed" it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file. Eg:

def admin_email
return 'expertiza-support@lists.ncsu.edu' if heroku_pr_review_app?

"expertiza-support@lists.ncsu.edu"
end

def admin_pass
return 'errbit' if heroku_pr_review_app?

@admin_pass ||= "errbit"
end

*Seed the DB (NOTE: No bootstrap task is used on Heroku!).
heroku run rake db:seed

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from your console
heroku login
*Added remote
heroku git:remote -a errbit-expertiza2019
*As errbit needs MongoDB database backend so need to install MongoDB addon in Heroku. We can use heroku command to create addon one.
heroku addons:add mongolab:sandbox 
heroku config:add HEROKU=true 

git add . 
git commit -m “new creds” 
git push heroku master 

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy it's configuration and place it in the expertiza/config/initializers/airbrake.rb file.
Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

Independent Study Spring 2019/Errbit

2019-05-08T05:49:19Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run rake errbit:bootsrap or rake db:seed it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file.

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from console using `heroku login`
*Added remote using `heroku git:remote -a errbit-expertiza2019`

heroku addons:add mongolab:sandbox 
heroku addons:add sendgrid:starter 
heroku config:add HEROKU=true 

git add . 
git commit -m “new creds” 
git push heroku master 

'''Configure Expertiza to point to our deployed Errbit application
*Create a new app on Errbit
*Copy it's configuration and place it in the expertiza/config/initializers/airbrake.rb file.
Airbrake.configure do |config|
config.host = 'https://errbit-expertiza2019.herokuapp.com'
config.project_id = 1 # required, but any positive integer works
config.project_key = '64ed97f0c8e628acefb3a7f63308a11c'
# Uncomment for Rails apps
config.environment = Rails.env
config.ignore_environments = %w(test)
end

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

Independent Study Spring 2019/Errbit

2019-05-07T21:29:59Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run rake errbit:bootsrap or rake db:seed it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file.

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from console using `heroku login`
*Added remote using `heroku git:remote -a errbit-expertiza2019`

heroku addons:add mongolab:sandbox 
heroku addons:add sendgrid:starter 
heroku config:add HEROKU=true 

> git add .
> git commit -m “new creds”
> git push heroku master

'''Configure Expertiza to send errors to the deployed Errbit
*Create a new app on Errbit
*Copy it's configuration and place it in Expertiza

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

Independent Study Spring 2019/Errbit

2019-05-07T21:12:03Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run rake errbit:bootsrap or rake db:seed it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file.

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from console using `heroku login`
*Added remote using `heroku git:remote -a errbit-expertiza2019`
```
heroku addons:add mongolab:sandbox
heroku addons:add sendgrid:starter
heroku config:add HEROKU=true
```
> git add .
> git commit -m “new creds”
> git push heroku master

'''Configure Expertiza to send errors to the deployed Errbit
*Create a new app on Errbit
*Copy it's configuration and place it in Expertiza

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

Independent Study Spring 2019/Errbit

2019-05-07T21:10:38Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run rake errbit:bootsrap or rake db:seed it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file.

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, named errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git

Took the following steps in the /errbit repository on local machine:

*Login to Heroku using Herkou CLI from console using `heroku login`
*Added remote using `heroku git:remote -a errbit-expertiza2019`
```
heroku addons:add mongolab:sandbox
heroku addons:add sendgrid:starter
heroku config:add HEROKU=true
```
> git add .
> git commit -m “new creds”
> git push heroku master

'''Configure Expertiza to send errors to the deployed Errbit
*Create a new app on Errbit
*Copy it's configuration and place it in Expertiza

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

==Log files==
For different log levels, log messages are written in different files named expertiza_<log_level>.log

Currently, expertiza provides the capability of writing logs into 5 different custom log files.

For eg,

*expertiza_info.log

*expertiza_warn.log

*expertiza_error.log

*expertiza_fatal.log

*expertiza_debug.log

Apart from above files, <environment>.log file is used for logging messages which could not be logged by ExpertizaLogger, like, page load messages or certain load time exceptions.

Since Rails logger provides writing into a specific log file, to restrict least file switch, ExpertizaLogger holds different log level loggers in separate instance variables. A new logger instance is created and assigned whenever Rails removes the link to manage memory consumption.

==Components==
===ExpertizaLogger===

This class provides the logger needed for logging messages for different log levels.
It provides 5 different static methods namely '''info''', '''warn''', '''error''', '''fatal''' and '''debug''' which could be called as follows:

'''ExpertizaLogger.info ''message'''''

where ''message'' could be an instance of '''LoggerMessage''' or a string.

===Custom LoggerMessage===

A LoggerMessage instance could be created as follows:

'''LoggerMessage.new(<controller_name>, <user_id>, <message>, <request object>)'''

Fourth parameter i.e <request object> is an optional parameter.

Whenever a request is made, the request object is injected into the controller and is available in the controller as "request".
Similarly, <controller_name> is available in "controller_name" inside controller. For other trigger points, file name could be passed as a string.

Since the request object holds the ''<origin_ip>'' and ''<request_id>'', it is used to capture and log both into the log message.

===ExpertizaLogFormatter===

It captures the message and determines if the message is an instance of LoggerMessage or not. Based on that, it creates a message by populating message placeholders as below:

if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end

All the exceptions are preprocessed to remove newline characters so that it could fit in a single line as a message.

All the messages that are not logged by ExpertizaLogger, there is a separate logger definition provided in config/environments/<environment>.rb as:

config.log_formatter = proc do |s, ts, pg, msg|
if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end
end

===Environment config===
config.log_tags = [ :remote_ip, :uuid ]

These config.log_tags entries make sure that <origin_ip> and <request_id> are prepended to messages. These are for messages which are not logged by ExpertizaLogger but by the Rails default logger.
These messages go in <environment>.log file.

==Restrictions==
The request object is not available beyond the controllers. Hence, ''<origin_ip>'' and ''<request_id>'' remains uncaptured if messages are logged from somewhere other than controllers.

==Visualization through ELK stack==
===Configuration===

Filebeat (filebeat.yml)

Changes:

- type: log

# Change to true to enable this prospector configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- <path to log files>*.log

output.logstash:
# The Logstash hosts
hosts: ["<ip>:<5044:port of logstash>"]

Filebeat sends all log files to logstash.

Logstash (expertiza-logstash.conf)

input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "TST=\[%{DATA:timestamp}\] SVT=\[%{DATA:sev}\] PNM=\[%{DATA:pgnm}\] OIP=\[%{DATA:originip}\] RID=\[%{DATA:requestid}\] CTR=\[%{DATA:generatedby}\] UID=\[%
{DATA:unityid}\] MSG=\[%{GREEDYDATA:message}\]" }
overwrite => [ "message", "timestamp" ]
}
date {
match => ["timestamp", "ISO8601"]
}

}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}

Logstash reads each line from the log file and parses it based on the template provided in grok match above. It then sends each message entry into Elastic search.

Kibana (config/kibana.yml)

elasticsearch.url: "http://localhost:9200"

Kibana picks each entry from Elastic search and provides into the web interface.

===Setup===
ELK stack could be set up to visualize and analyze the messages from the log files.

Once ELK is installed, below commands could be used to run each server:

'''Clear Elastic search cache'''

curl -X DELETE 'http://localhost:9200/_all'

'''Filebeat'''

sudo rm data/registry

sudo ./filebeat -e -c filebeat.yml -d "publish"

'''Logstash'''

bin/logstash -f expertiza-logstash.conf --config.reload.automatic

'''Kibana'''

bin/kibana

===Kibana Interface===

Success message in Kibana

[[File:kibana_success_msg.png|frame|upright|center]]

Exception in kibana

[[File:kibana_exception.png|frame|upright|center]]

Tracing error in kibana

[[File:kibana_tracing_error.png|frame|upright|center]]

Search records using request ID in kibana

[[File:kibana_search_req_id.png|frame|upright|center]]

Searching records using unity id in kibana

[[File:kibana_search_using_unityid.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-07T21:00:15Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with Email: expertiza-support@lists.ncsu.edu and Password: expertiza2019@

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit and follow the steps mentioned on [https://github.com/errbit/errbit/blob/master/README.md README.md]
*When we run rake errbit:bootsrap or rake db:seed it will create an admin user with a random password. We can see these login credential in the console log. But instead, we can provide this username and password explicitly by just making some changes in errbit/db/seed.rb file.

'''Deploy Errbit on Heroku'''
*Created a new empty application on Heroku, namely errbit-expertiza2019
*The app will have an associated empty git repository https://git.heroku.com/errbit-expertiza2019.git
*Login to Heroku using Herkou CLI
*

'''Configure Expertiza to send errors to the deployed Errbit
*Create a new app on Errbit
*Copy it's configuration and place it in Expertiza

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.'''

==Log files==
For different log levels, log messages are written in different files named expertiza_<log_level>.log

Currently, expertiza provides the capability of writing logs into 5 different custom log files.

For eg,

*expertiza_info.log

*expertiza_warn.log

*expertiza_error.log

*expertiza_fatal.log

*expertiza_debug.log

Apart from above files, <environment>.log file is used for logging messages which could not be logged by ExpertizaLogger, like, page load messages or certain load time exceptions.

Since Rails logger provides writing into a specific log file, to restrict least file switch, ExpertizaLogger holds different log level loggers in separate instance variables. A new logger instance is created and assigned whenever Rails removes the link to manage memory consumption.

==Components==
===ExpertizaLogger===

This class provides the logger needed for logging messages for different log levels.
It provides 5 different static methods namely '''info''', '''warn''', '''error''', '''fatal''' and '''debug''' which could be called as follows:

'''ExpertizaLogger.info ''message'''''

where ''message'' could be an instance of '''LoggerMessage''' or a string.

===Custom LoggerMessage===

A LoggerMessage instance could be created as follows:

'''LoggerMessage.new(<controller_name>, <user_id>, <message>, <request object>)'''

Fourth parameter i.e <request object> is an optional parameter.

Whenever a request is made, the request object is injected into the controller and is available in the controller as "request".
Similarly, <controller_name> is available in "controller_name" inside controller. For other trigger points, file name could be passed as a string.

Since the request object holds the ''<origin_ip>'' and ''<request_id>'', it is used to capture and log both into the log message.

===ExpertizaLogFormatter===

It captures the message and determines if the message is an instance of LoggerMessage or not. Based on that, it creates a message by populating message placeholders as below:

if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end

All the exceptions are preprocessed to remove newline characters so that it could fit in a single line as a message.

All the messages that are not logged by ExpertizaLogger, there is a separate logger definition provided in config/environments/<environment>.rb as:

config.log_formatter = proc do |s, ts, pg, msg|
if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end
end

===Environment config===
config.log_tags = [ :remote_ip, :uuid ]

These config.log_tags entries make sure that <origin_ip> and <request_id> are prepended to messages. These are for messages which are not logged by ExpertizaLogger but by the Rails default logger.
These messages go in <environment>.log file.

==Restrictions==
The request object is not available beyond the controllers. Hence, ''<origin_ip>'' and ''<request_id>'' remains uncaptured if messages are logged from somewhere other than controllers.

==Visualization through ELK stack==
===Configuration===

Filebeat (filebeat.yml)

Changes:

- type: log

# Change to true to enable this prospector configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- <path to log files>*.log

output.logstash:
# The Logstash hosts
hosts: ["<ip>:<5044:port of logstash>"]

Filebeat sends all log files to logstash.

Logstash (expertiza-logstash.conf)

input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "TST=\[%{DATA:timestamp}\] SVT=\[%{DATA:sev}\] PNM=\[%{DATA:pgnm}\] OIP=\[%{DATA:originip}\] RID=\[%{DATA:requestid}\] CTR=\[%{DATA:generatedby}\] UID=\[%
{DATA:unityid}\] MSG=\[%{GREEDYDATA:message}\]" }
overwrite => [ "message", "timestamp" ]
}
date {
match => ["timestamp", "ISO8601"]
}

}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}

Logstash reads each line from the log file and parses it based on the template provided in grok match above. It then sends each message entry into Elastic search.

Kibana (config/kibana.yml)

elasticsearch.url: "http://localhost:9200"

Kibana picks each entry from Elastic search and provides into the web interface.

===Setup===
ELK stack could be set up to visualize and analyze the messages from the log files.

Once ELK is installed, below commands could be used to run each server:

'''Clear Elastic search cache'''

curl -X DELETE 'http://localhost:9200/_all'

'''Filebeat'''

sudo rm data/registry

sudo ./filebeat -e -c filebeat.yml -d "publish"

'''Logstash'''

bin/logstash -f expertiza-logstash.conf --config.reload.automatic

'''Kibana'''

bin/kibana

===Kibana Interface===

Success message in Kibana

[[File:kibana_success_msg.png|frame|upright|center]]

Exception in kibana

[[File:kibana_exception.png|frame|upright|center]]

Tracing error in kibana

[[File:kibana_tracing_error.png|frame|upright|center]]

Search records using request ID in kibana

[[File:kibana_search_req_id.png|frame|upright|center]]

Searching records using unity id in kibana

[[File:kibana_search_using_unityid.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-07T20:24:16Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with email: expertiza-support@lists.ncsu.edu and password: expertiza2019@
*

'''Setup Errbit'''
*Fork Errbit from https://github.com/errbit/errbit
*Make changes to login using own creds
*Make an app on Errbit
*Copy it's configuration and place it in Expertiza

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.
==Log files==
For different log levels, log messages are written in different files named expertiza_<log_level>.log

Currently, expertiza provides the capability of writing logs into 5 different custom log files.

For eg,

*expertiza_info.log

*expertiza_warn.log

*expertiza_error.log

*expertiza_fatal.log

*expertiza_debug.log

Apart from above files, <environment>.log file is used for logging messages which could not be logged by ExpertizaLogger, like, page load messages or certain load time exceptions.

Since Rails logger provides writing into a specific log file, to restrict least file switch, ExpertizaLogger holds different log level loggers in separate instance variables. A new logger instance is created and assigned whenever Rails removes the link to manage memory consumption.

==Components==
===ExpertizaLogger===

This class provides the logger needed for logging messages for different log levels.
It provides 5 different static methods namely '''info''', '''warn''', '''error''', '''fatal''' and '''debug''' which could be called as follows:

'''ExpertizaLogger.info ''message'''''

where ''message'' could be an instance of '''LoggerMessage''' or a string.

===Custom LoggerMessage===

A LoggerMessage instance could be created as follows:

'''LoggerMessage.new(<controller_name>, <user_id>, <message>, <request object>)'''

Fourth parameter i.e <request object> is an optional parameter.

Whenever a request is made, the request object is injected into the controller and is available in the controller as "request".
Similarly, <controller_name> is available in "controller_name" inside controller. For other trigger points, file name could be passed as a string.

Since the request object holds the ''<origin_ip>'' and ''<request_id>'', it is used to capture and log both into the log message.

===ExpertizaLogFormatter===

It captures the message and determines if the message is an instance of LoggerMessage or not. Based on that, it creates a message by populating message placeholders as below:

if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end

All the exceptions are preprocessed to remove newline characters so that it could fit in a single line as a message.

All the messages that are not logged by ExpertizaLogger, there is a separate logger definition provided in config/environments/<environment>.rb as:

config.log_formatter = proc do |s, ts, pg, msg|
if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end
end

===Environment config===
config.log_tags = [ :remote_ip, :uuid ]

These config.log_tags entries make sure that <origin_ip> and <request_id> are prepended to messages. These are for messages which are not logged by ExpertizaLogger but by the Rails default logger.
These messages go in <environment>.log file.

==Restrictions==
The request object is not available beyond the controllers. Hence, ''<origin_ip>'' and ''<request_id>'' remains uncaptured if messages are logged from somewhere other than controllers.

==Visualization through ELK stack==
===Configuration===

Filebeat (filebeat.yml)

Changes:

- type: log

# Change to true to enable this prospector configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- <path to log files>*.log

output.logstash:
# The Logstash hosts
hosts: ["<ip>:<5044:port of logstash>"]

Filebeat sends all log files to logstash.

Logstash (expertiza-logstash.conf)

input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "TST=\[%{DATA:timestamp}\] SVT=\[%{DATA:sev}\] PNM=\[%{DATA:pgnm}\] OIP=\[%{DATA:originip}\] RID=\[%{DATA:requestid}\] CTR=\[%{DATA:generatedby}\] UID=\[%
{DATA:unityid}\] MSG=\[%{GREEDYDATA:message}\]" }
overwrite => [ "message", "timestamp" ]
}
date {
match => ["timestamp", "ISO8601"]
}

}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}

Logstash reads each line from the log file and parses it based on the template provided in grok match above. It then sends each message entry into Elastic search.

Kibana (config/kibana.yml)

elasticsearch.url: "http://localhost:9200"

Kibana picks each entry from Elastic search and provides into the web interface.

===Setup===
ELK stack could be set up to visualize and analyze the messages from the log files.

Once ELK is installed, below commands could be used to run each server:

'''Clear Elastic search cache'''

curl -X DELETE 'http://localhost:9200/_all'

'''Filebeat'''

sudo rm data/registry

sudo ./filebeat -e -c filebeat.yml -d "publish"

'''Logstash'''

bin/logstash -f expertiza-logstash.conf --config.reload.automatic

'''Kibana'''

bin/kibana

===Kibana Interface===

Success message in Kibana

[[File:kibana_success_msg.png|frame|upright|center]]

Exception in kibana

[[File:kibana_exception.png|frame|upright|center]]

Tracing error in kibana

[[File:kibana_tracing_error.png|frame|upright|center]]

Search records using request ID in kibana

[[File:kibana_search_req_id.png|frame|upright|center]]

Searching records using unity id in kibana

[[File:kibana_search_using_unityid.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-07T20:11:49Z

Psingh25:

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Errbit has been set-up, replacing the already existing Airbrake API to serve several purposes like :
*Errbit is a much more powerful monitoring tool
*Errbit is Airbrake API compliant
*Store unresolved errors indefinitely without any extra cost.

==The Steps taken for the setup==
There were several steps taken to set-up Errbit for Expertiza and schedule it to automatically pull changed from its forked branch and deploy:

'''Setup Heroku'''
*Made an account with email: and password:
*

'''Setup Errbit'''
*Fork Errbit
*Make changes to login using own creds
*Make an app on Errbit
*Copy it's configuration and place it in Expertiza

'''Set up Cron job to regularly pull changes from Forked repository and deploy on Heroku.
==Log files==
For different log levels, log messages are written in different files named expertiza_<log_level>.log

Currently, expertiza provides the capability of writing logs into 5 different custom log files.

For eg,

*expertiza_info.log

*expertiza_warn.log

*expertiza_error.log

*expertiza_fatal.log

*expertiza_debug.log

Apart from above files, <environment>.log file is used for logging messages which could not be logged by ExpertizaLogger, like, page load messages or certain load time exceptions.

Since Rails logger provides writing into a specific log file, to restrict least file switch, ExpertizaLogger holds different log level loggers in separate instance variables. A new logger instance is created and assigned whenever Rails removes the link to manage memory consumption.

==Components==
===ExpertizaLogger===

This class provides the logger needed for logging messages for different log levels.
It provides 5 different static methods namely '''info''', '''warn''', '''error''', '''fatal''' and '''debug''' which could be called as follows:

'''ExpertizaLogger.info ''message'''''

where ''message'' could be an instance of '''LoggerMessage''' or a string.

===Custom LoggerMessage===

A LoggerMessage instance could be created as follows:

'''LoggerMessage.new(<controller_name>, <user_id>, <message>, <request object>)'''

Fourth parameter i.e <request object> is an optional parameter.

Whenever a request is made, the request object is injected into the controller and is available in the controller as "request".
Similarly, <controller_name> is available in "controller_name" inside controller. For other trigger points, file name could be passed as a string.

Since the request object holds the ''<origin_ip>'' and ''<request_id>'', it is used to capture and log both into the log message.

===ExpertizaLogFormatter===

It captures the message and determines if the message is an instance of LoggerMessage or not. Based on that, it creates a message by populating message placeholders as below:

if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end

All the exceptions are preprocessed to remove newline characters so that it could fit in a single line as a message.

All the messages that are not logged by ExpertizaLogger, there is a separate logger definition provided in config/environments/<environment>.rb as:

config.log_formatter = proc do |s, ts, pg, msg|
if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end
end

===Environment config===
config.log_tags = [ :remote_ip, :uuid ]

These config.log_tags entries make sure that <origin_ip> and <request_id> are prepended to messages. These are for messages which are not logged by ExpertizaLogger but by the Rails default logger.
These messages go in <environment>.log file.

==Restrictions==
The request object is not available beyond the controllers. Hence, ''<origin_ip>'' and ''<request_id>'' remains uncaptured if messages are logged from somewhere other than controllers.

==Visualization through ELK stack==
===Configuration===

Filebeat (filebeat.yml)

Changes:

- type: log

# Change to true to enable this prospector configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- <path to log files>*.log

output.logstash:
# The Logstash hosts
hosts: ["<ip>:<5044:port of logstash>"]

Filebeat sends all log files to logstash.

Logstash (expertiza-logstash.conf)

input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "TST=\[%{DATA:timestamp}\] SVT=\[%{DATA:sev}\] PNM=\[%{DATA:pgnm}\] OIP=\[%{DATA:originip}\] RID=\[%{DATA:requestid}\] CTR=\[%{DATA:generatedby}\] UID=\[%
{DATA:unityid}\] MSG=\[%{GREEDYDATA:message}\]" }
overwrite => [ "message", "timestamp" ]
}
date {
match => ["timestamp", "ISO8601"]
}

}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}

Logstash reads each line from the log file and parses it based on the template provided in grok match above. It then sends each message entry into Elastic search.

Kibana (config/kibana.yml)

elasticsearch.url: "http://localhost:9200"

Kibana picks each entry from Elastic search and provides into the web interface.

===Setup===
ELK stack could be set up to visualize and analyze the messages from the log files.

Once ELK is installed, below commands could be used to run each server:

'''Clear Elastic search cache'''

curl -X DELETE 'http://localhost:9200/_all'

'''Filebeat'''

sudo rm data/registry

sudo ./filebeat -e -c filebeat.yml -d "publish"

'''Logstash'''

bin/logstash -f expertiza-logstash.conf --config.reload.automatic

'''Kibana'''

bin/kibana

===Kibana Interface===

Success message in Kibana

[[File:kibana_success_msg.png|frame|upright|center]]

Exception in kibana

[[File:kibana_exception.png|frame|upright|center]]

Tracing error in kibana

[[File:kibana_tracing_error.png|frame|upright|center]]

Search records using request ID in kibana

[[File:kibana_search_req_id.png|frame|upright|center]]

Searching records using unity id in kibana

[[File:kibana_search_using_unityid.png|frame|upright|center]]

Independent Study Spring 2019/Errbit

2019-05-07T19:59:40Z

Psingh25: Created page with "=Set up an open-source error monitoring tool instead of Airbrake= ==Useful links== [https://github.com/expertiza/expertiza/pull/1148 Github Pull Request] [https://github.com/err..."

=Set up an open-source error monitoring tool instead of Airbrake=
==Useful links==
[https://github.com/expertiza/expertiza/pull/1148 Github Pull Request]

[https://github.com/errbit/errbit Errbit]

==Purpose==
Logging feature has been added to Expertiza through a custom logging module written uniquely to serve several purposes such as:
*Tracing application failure
*Analysing user request flow
*Managing different logging level

==Message Format==
All the messages are logged in either of the below formats:

'''TST=[2018-03-10 00:50:18 -0500] SVT=[<''log_level''>] PNM=[<''program_name''>] OIP=[<''origin_ip''>] RID=[<''request_id''>] CTR=[<''controller_name''>] UID=[<''user_id''>] MSG=[''Message'']'''

or

'''TST=[2018-03-10 00:50:18 -0500] SVT=[<''log_level''>] PNM=[<''program_name''>] OIP=[] RID=[] CTR=[<''controller_name''>] UID=[<''user_id''>] MSG=[[''origin_ip''] [''request_id''] ''Message'']'''

Above variation is because when certain messages are logged, ExpertizaLogger is unaware of origin_ip and request_ip and thus it is appended to the message instead of their respective placeholders.

===Meaning of each placeholder===
'''SVT''' - It denotes the severity level of the log. For eg, if the logging is done by calling logger.info, the severity level is "INFO".

'''PNM''' - It's the program name and is same as provided by default Rails logger.

'''OIP''' - It denotes the IP address of the system from where the request has been made.

'''RID''' - It's the ID of the request. All subsequent calls made by the client till the actual response is returned will have same request ID. For eg, when a page is first requested, all smaller page template fetched, all method flow will have same request ID. This field can be used to understand how a certain request was served.

'''CTR''' - It is the controller name from where the logging happened.

'''UID''' - It is the unity id/user id of the user making requests. This field could also be used in identifying the cause of an error.

'''Message''' - It is the actual message logged.

==Log files==
For different log levels, log messages are written in different files named expertiza_<log_level>.log

Currently, expertiza provides the capability of writing logs into 5 different custom log files.

For eg,

*expertiza_info.log

*expertiza_warn.log

*expertiza_error.log

*expertiza_fatal.log

*expertiza_debug.log

Apart from above files, <environment>.log file is used for logging messages which could not be logged by ExpertizaLogger, like, page load messages or certain load time exceptions.

Since Rails logger provides writing into a specific log file, to restrict least file switch, ExpertizaLogger holds different log level loggers in separate instance variables. A new logger instance is created and assigned whenever Rails removes the link to manage memory consumption.

==Components==
===ExpertizaLogger===

This class provides the logger needed for logging messages for different log levels.
It provides 5 different static methods namely '''info''', '''warn''', '''error''', '''fatal''' and '''debug''' which could be called as follows:

'''ExpertizaLogger.info ''message'''''

where ''message'' could be an instance of '''LoggerMessage''' or a string.

===Custom LoggerMessage===

A LoggerMessage instance could be created as follows:

'''LoggerMessage.new(<controller_name>, <user_id>, <message>, <request object>)'''

Fourth parameter i.e <request object> is an optional parameter.

Whenever a request is made, the request object is injected into the controller and is available in the controller as "request".
Similarly, <controller_name> is available in "controller_name" inside controller. For other trigger points, file name could be passed as a string.

Since the request object holds the ''<origin_ip>'' and ''<request_id>'', it is used to capture and log both into the log message.

===ExpertizaLogFormatter===

It captures the message and determines if the message is an instance of LoggerMessage or not. Based on that, it creates a message by populating message placeholders as below:

if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end

All the exceptions are preprocessed to remove newline characters so that it could fit in a single line as a message.

All the messages that are not logged by ExpertizaLogger, there is a separate logger definition provided in config/environments/<environment>.rb as:

config.log_formatter = proc do |s, ts, pg, msg|
if msg.is_a?(LoggerMessage)
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[#{msg.oip}] RID=[#{msg.req_id}] CTR=[#{msg.generator}] UID=[#{msg.unity_id}] MSG=[#{filter(msg.message)}]\n"
else
"TST=[#{ts}] SVT=[#{s}] PNM=[#{pg}] OIP=[] RID=[] CTR=[] UID=[] MSG=[#{filter(msg)}]\n"
end
end

===Environment config===
config.log_tags = [ :remote_ip, :uuid ]

These config.log_tags entries make sure that <origin_ip> and <request_id> are prepended to messages. These are for messages which are not logged by ExpertizaLogger but by the Rails default logger.
These messages go in <environment>.log file.

==Restrictions==
The request object is not available beyond the controllers. Hence, ''<origin_ip>'' and ''<request_id>'' remains uncaptured if messages are logged from somewhere other than controllers.

==Visualization through ELK stack==
===Configuration===

Filebeat (filebeat.yml)

Changes:

- type: log

# Change to true to enable this prospector configuration.
enabled: true

# Paths that should be crawled and fetched. Glob based paths.
paths:
- <path to log files>*.log

output.logstash:
# The Logstash hosts
hosts: ["<ip>:<5044:port of logstash>"]

Filebeat sends all log files to logstash.

Logstash (expertiza-logstash.conf)

input {
beats {
port => "5044"
}
}
filter {
grok {
match => { "message" => "TST=\[%{DATA:timestamp}\] SVT=\[%{DATA:sev}\] PNM=\[%{DATA:pgnm}\] OIP=\[%{DATA:originip}\] RID=\[%{DATA:requestid}\] CTR=\[%{DATA:generatedby}\] UID=\[%
{DATA:unityid}\] MSG=\[%{GREEDYDATA:message}\]" }
overwrite => [ "message", "timestamp" ]
}
date {
match => ["timestamp", "ISO8601"]
}

}
output {
elasticsearch {
hosts => [ "localhost:9200" ]
}
}

Logstash reads each line from the log file and parses it based on the template provided in grok match above. It then sends each message entry into Elastic search.

Kibana (config/kibana.yml)

elasticsearch.url: "http://localhost:9200"

Kibana picks each entry from Elastic search and provides into the web interface.

===Setup===
ELK stack could be set up to visualize and analyze the messages from the log files.

Once ELK is installed, below commands could be used to run each server:

'''Clear Elastic search cache'''

curl -X DELETE 'http://localhost:9200/_all'

'''Filebeat'''

sudo rm data/registry

sudo ./filebeat -e -c filebeat.yml -d "publish"

'''Logstash'''

bin/logstash -f expertiza-logstash.conf --config.reload.automatic

'''Kibana'''

bin/kibana

===Kibana Interface===

Success message in Kibana

[[File:kibana_success_msg.png|frame|upright|center]]

Exception in kibana

[[File:kibana_exception.png|frame|upright|center]]

Tracing error in kibana

[[File:kibana_tracing_error.png|frame|upright|center]]

Search records using request ID in kibana

[[File:kibana_search_req_id.png|frame|upright|center]]

Searching records using unity id in kibana

[[File:kibana_search_using_unityid.png|frame|upright|center]]

User:Psingh25

2019-05-05T19:41:06Z

Psingh25:

<h1> Independent Study Wiki </h1>

User:Psingh25

2019-05-05T19:40:54Z

Psingh25: Created page with "<h1> Independent Study Wiki <h1/>"

<h1> Independent Study Wiki <h1/>

CSC517 OffScreenCanvasContext2D

2018-11-23T22:56:00Z

Psingh25:

== For reviewers ==

Please go through the following wiki page first, since this project is a continuation of [http://wiki.expertiza.ncsu.edu/index.php/CSC517_OffScreenCanvas_Servo this] one.

== About Servo: ==

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except for the fact that it is specially tuned for better memory safety and concurrency features.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders, and constellation. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affect other tabs, browser or even other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

''Why use rust ?''

Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

== Before we get started ==
=== Off Screen Canvas: ===

Canvas is a popular way of drawing all kinds of graphics on the screen and an entry point to the world of WebGL. It can be used to draw shapes, images, run animations, or even display and process video content. It is often used to create beautiful user experiences in media-rich web applications and online games. Because canvas logic and rendering happens on the same thread as user interaction, the (sometimes heavy) computations involved in animations can harm the app’s real and perceived performance.

Until now, canvas drawing capabilities were tied to the <canvas> element, which meant it was directly depending on the DOM. OffscreenCanvas, as the name suggests, decouples the DOM and the Canvas API by moving it off-screen. Due to this decoupling, rendering of OffscreenCanvas is fully detached from the DOM and therefore offers some speed improvements over the regular canvas as there is no synchronization between the two.

===Web workers and Offscreen Canvas===

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages.
''But, how is OffscreenCanvas rendered using web-workers?:'' Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

=== Web IDL ===
Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an [[IDL specification language|IDL]] variant with:

* A number of features that allow one to more easily describe the behavior of common script objects in a web context.

* A mapping of how interfaces described with Web IDL correspond to language constructs within an [[ECMAScript]] execution environment.

== Problem Statement: ==

This document describes the final project of CSC 517 course, that involves merging a few changes into
servo, a parallel web browser engine developed by Mozilla. This project builds upon the previous
project under CSC, which was also to merge a few changes into servo. For further a detailed
information regarding the previous project, please visit:
http://wiki.expertiza.ncsu.edu/index.php/CSC517_OffScreenCanvas_Servo.

OffscreenCanvas API is an API which enables the rendering of canvas elements outside the current
thread of browser engine or webpage execution. OffscreenCanvas is exactly similar to HTML Canvas,
except the fact that it does not have access to DOM and window elements, and it is rendered on a
thread completely independent of the main web-page execution. Our task is to implement two
interfaces, OffscreenCanvas & OffscreenCanvasContextRendering2D, which would enable this
functionality. Then, if the changes made by us are successfully built, the next task is to enable
automated tests to check for the accuracy of the code. Finally, we’ll test our code by enabling the blob
API.

== Getting Started ==

== Building Servo ==
* Testing servo requires it to build it locally, or using a cloud service like [http://janitor.technology Janitor]. 
* Building servo locally would take approximately 30mins - 1 hour depending upon your computer. 
* For further steps regarding building servo locally, please visit [https://github.com/servo/servo this]. 

== Verifying Build ==
* We can verify whether servo has been built by running the following command: ./mach run http://www.google.com 
* This command will render the homepage of google in a new browser instance of servo. If this executes correctly, then the build is fine. 
* Specific functionalities could be tested by writing a custom html file and then running it the similar way. 
* Note: the following command won't work if you're on Janitor.

=== What we've done ===

As a part of the third project, we’ve already done with the implementation of OffscreenCanvas
interface. We’ve made the corresponding implementation of the webidl file using a rust file, and have
also enabled and passed the automated tests. We’ve also made the interface for
OffscreenCanvasContextRendering2D.

=== Proposed Design ===

As a part of the fourth project, we’ve planned to first make an implementation for the
OffscreenCanvasContextRendering2D file that we’ve created. This the the file which would perform all
the rendering in the back-end. Since OffscreenCanvas is very similar to HTML canvas, we’ll first copy
the code from HTMLCanvasContextRendering2D, and paste it into our implementation. We would
then filter out the code which is not a part of the OffscreenCanvasContextRendering2D, and then add
the functionalities that should be present. Finally, we would implement the blob API, which would
enable us to test the contents of the canvas. Then, similarly, we would support webgl contexts, in a
manner similar to the 2D context, by sharing relevant code between
OffscreenCanvasRenderingContext2D and WebGLRenderingContext.

== Files we've changed and to-be changed ==

1. OffscreenCanvas.webidl 
2. OffscreenCanvas.rs 
3. OffscreenCanvasContextRendering2D.webidl 
4. OffscreenCanvasCOntextRendering2D.rs 
5. resources/prefs.json 
6. tests/wpt/include.ini 
7. ___dir___.ini 

== References: ==

1. https://research.mozilla.org/servo-engines/ 
2. https://research.mozilla.org/rust/ 
3. https://hacks.mozilla.org/2017/05/quantum-up-close-what-is-a-browser-engine/

CSC517 OffScreenCanvasContext2D

2018-11-23T22:54:19Z

Psingh25:

== For reviewers ==

Please go through the following wiki page first, since this project is in continuation with [http://wiki.expertiza.ncsu.edu/index.php/CSC517_OffScreenCanvas_Servo this] previous one.

== About Servo: ==

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except for the fact that it is specially tuned for better memory safety and concurrency features.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders, and constellation. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affect other tabs, browser or even other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

''Why use rust ?''

Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

== Before we get started ==
=== Off Screen Canvas: ===

Canvas is a popular way of drawing all kinds of graphics on the screen and an entry point to the world of WebGL. It can be used to draw shapes, images, run animations, or even display and process video content. It is often used to create beautiful user experiences in media-rich web applications and online games. Because canvas logic and rendering happens on the same thread as user interaction, the (sometimes heavy) computations involved in animations can harm the app’s real and perceived performance.

Until now, canvas drawing capabilities were tied to the <canvas> element, which meant it was directly depending on the DOM. OffscreenCanvas, as the name suggests, decouples the DOM and the Canvas API by moving it off-screen. Due to this decoupling, rendering of OffscreenCanvas is fully detached from the DOM and therefore offers some speed improvements over the regular canvas as there is no synchronization between the two.

===Web workers and Offscreen Canvas===

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages.
''But, how is OffscreenCanvas rendered using web-workers?:'' Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

=== Web IDL ===
Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an [[IDL specification language|IDL]] variant with:

* A number of features that allow one to more easily describe the behavior of common script objects in a web context.

* A mapping of how interfaces described with Web IDL correspond to language constructs within an [[ECMAScript]] execution environment.

== Problem Statement: ==

This document describes the final project of CSC 517 course, that involves merging a few changes into
servo, a parallel web browser engine developed by Mozilla. This project builds upon the previous
project under CSC, which was also to merge a few changes into servo. For further a detailed
information regarding the previous project, please visit:
http://wiki.expertiza.ncsu.edu/index.php/CSC517_OffScreenCanvas_Servo.

OffscreenCanvas API is an API which enables the rendering of canvas elements outside the current
thread of browser engine or webpage execution. OffscreenCanvas is exactly similar to HTML Canvas,
except the fact that it does not have access to DOM and window elements, and it is rendered on a
thread completely independent of the main web-page execution. Our task is to implement two
interfaces, OffscreenCanvas & OffscreenCanvasContextRendering2D, which would enable this
functionality. Then, if the changes made by us are successfully built, the next task is to enable
automated tests to check for the accuracy of the code. Finally, we’ll test our code by enabling the blob
API.

== Getting Started ==

== Building Servo ==
* Testing servo requires it to build it locally, or using a cloud service like [http://janitor.technology Janitor]. 
* Building servo locally would take approximately 30mins - 1 hour depending upon your computer. 
* For further steps regarding building servo locally, please visit [https://github.com/servo/servo this]. 

== Verifying Build ==
* We can verify whether servo has been built by running the following command: ./mach run http://www.google.com 
* This command will render the homepage of google in a new browser instance of servo. If this executes correctly, then the build is fine. 
* Specific functionalities could be tested by writing a custom html file and then running it the similar way. 
* Note: the following command won't work if you're on Janitor.

=== What we've done ===

As a part of the third project, we’ve already done with the implementation of OffscreenCanvas
interface. We’ve made the corresponding implementation of the webidl file using a rust file, and have
also enabled and passed the automated tests. We’ve also made the interface for
OffscreenCanvasContextRendering2D.

=== Proposed Design ===

As a part of the fourth project, we’ve planned to first make an implementation for the
OffscreenCanvasContextRendering2D file that we’ve created. This the the file which would perform all
the rendering in the back-end. Since OffscreenCanvas is very similar to HTML canvas, we’ll first copy
the code from HTMLCanvasContextRendering2D, and paste it into our implementation. We would
then filter out the code which is not a part of the OffscreenCanvasContextRendering2D, and then add
the functionalities that should be present. Finally, we would implement the blob API, which would
enable us to test the contents of the canvas. Then, similarly, we would support webgl contexts, in a
manner similar to the 2D context, by sharing relevant code between
OffscreenCanvasRenderingContext2D and WebGLRenderingContext.

== Files we've changed and to-be changed ==

1. OffscreenCanvas.webidl 
2. OffscreenCanvas.rs 
3. OffscreenCanvasContextRendering2D.webidl 
4. OffscreenCanvasCOntextRendering2D.rs 
5. resources/prefs.json 
6. tests/wpt/include.ini 
7. ___dir___.ini 

== References: ==

1. https://research.mozilla.org/servo-engines/ 
2. https://research.mozilla.org/rust/ 
3. https://hacks.mozilla.org/2017/05/quantum-up-close-what-is-a-browser-engine/

CSC517 OffScreenCanvasContext2D

2018-11-14T04:16:07Z

Psingh25: /* What we've done */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:15:51Z

Psingh25: /* Files we've changed */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:15:01Z

Psingh25: /* Proposed Design */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:14:49Z

Psingh25: /* Verifying Build */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:14:36Z

Psingh25: /* Building Servo */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:14:02Z

Psingh25: /* Files we've changed */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:13:44Z

Psingh25: /* Files we've changed */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:12:05Z

Psingh25: /* What you've done */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:10:37Z

Psingh25: /* Before you get started */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:09:15Z

Psingh25: /* What we've planned to do */

CSC517 OffScreenCanvasContext2D

2018-11-14T04:05:22Z

Psingh25:

CSC517 OffScreenCanvasContext2D

2018-11-14T04:04:16Z

Psingh25:

CSC517 OffScreenCanvasContext2D

2018-11-14T02:56:13Z

Psingh25: Created page with "== About Servo: == Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse..."

CSC517 OffScreenCanvas Servo

2018-11-11T01:38:20Z

Psingh25: /* Under the hood */

== About Servo: ==

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except for the fact that it is specially tuned for better memory safety and concurrency features.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders, and constellation. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affect other tabs, browser or even other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

''Why use rust ?''

Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

== Before you get started ==
=== Off Screen Canvas: ===

Canvas is a popular way of drawing all kinds of graphics on the screen and an entry point to the world of WebGL. It can be used to draw shapes, images, run animations, or even display and process video content. It is often used to create beautiful user experiences in media-rich web applications and online games. Because canvas logic and rendering happens on the same thread as user interaction, the (sometimes heavy) computations involved in animations can harm the app’s real and perceived performance.

Until now, canvas drawing capabilities were tied to the <canvas> element, which meant it was directly depending on the DOM. OffscreenCanvas, as the name suggests, decouples the DOM and the Canvas API by moving it off-screen. Due to this decoupling, rendering of OffscreenCanvas is fully detached from the DOM and therefore offers some speed improvements over the regular canvas as there is no synchronization between the two.

===Web workers and Offscreen Canvas===

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages.
''But, how is OffscreenCanvas rendered using web-workers?:'' Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

=== Web IDL ===
Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an [[IDL specification language|IDL]] variant with:

* A number of features that allow one to more easily describe the behavior of common script objects in a web context.

* A mapping of how interfaces described with Web IDL correspond to language constructs within an [[ECMAScript]] execution environment.

== Scope: ==

The scope of the project included:

* Creating the OffscreenCanvas and OffscreenCanvasRenderingContext2d interfaces with stub method implementations.

* Implementing the OffscreenCanvas Constructor [https://html.spec.whatwg.org/multipage/scripting.html#dom-offscreencanvas] that creates a new canvas.

== Getting Started ==

=== Building Servo ===
* Testing servo requires it to build it locally, or using a cloud service like [http://janitor.technology Janitor]. 
* Building servo locally would take approximately 30mins - 1 hour depending upon your computer. 
* For further steps regarding building servo locally, please visit [https://github.com/servo/servo this]. 

=== Verifying Build ===
* We can verify whether servo has been built by running the following command: ./mach run http://www.google.com 
* This command will render the homepage of google in a new browser instance of servo. If this executes correctly, then the build is fine. 
* Specific functionalities could be tested by writing a custom html file and then running it the similar way. 
* Note: the following command won't work if you're on Janitor.

== What we did ==

=== Overview ===
* Our task was to implement the functionality of OffscreenCanvas. 
* Now, since OffscreenCanvas objects are declared and used in JavaScript, but the rendering happens outside the domain of JavaScript, we would need two different objects, one to interact with JavaScript, and the other to handle the rendering, independent of JavaScript. 
* The object to interact with JavaScript is OffscreenCanvas, whereas the object which handles rendering is OffscreenCanvasRenderingContext2D. 
* To implement a functionality, first we created the webIDL files, which contains interfaces of the functionalities we need to implement. 
* After creating the interfaces, the next step was to implement them. The implementation of the webIDL files is written in the rust file (.rs) with the same name. (For example, OffscreenCanvas.webidl -> offscreencanvas.rs) 
* The webIDL files contains the constructor as well as all the methods which were supposed to be implemented as specified in the webIDL files. 
* Testing the implemented code is discussed in further sections.

=== Code Flow ===
[[File:Workflow2108.png]]

=== Under the hood ===
* '''OffscreenCanvas.webidl:''' This is an interface, which defines what the implementation of the OffscreenCanvas should contain. As it directly interacts with JavaScript, hence it should have a constructor, with two arguments height and width, just like a regular HTMLCanvas element. The code for the same is shown in the implementation. Also, it contains the ''getContext'' method, which specifies the type of context: 2d, webgl or webgl2. 
* '''OffscreenCanvasRenderingContext2D.webidl:''' This is also an interface file, which defines what the implementation of OffscreenCanvasRenderingContext2D should look like. As it does not interacts directly with javascript, but interacts with the OffscreenCanvas object that we created, it contains a back reference to the same. It also defines the functionalities which its implementation class should follow.
* '''offscreencanvas.rs:''' This file is responsible for implementing OffscreenCanvas.webidl. It has a constructor, which is implemented as per the requirements of the webIDL, posted in the section below (implementation). When a new instance of offscreencanvas is created, it calls the constructor, which in turn calls the new method in the interface,\ which ultimately calls the new method defined in this file. The arguments of the constructor, namely height, width and (weak reference) options, are stored in the new object which is created. It also contains a ''reflector'' object, which tells the JavaScript garbage collector to ignore the canvas object while clearing memory. The implementation for the method ''getContext()'', returns a weak reference to the OffscreenCanvasRenderingContext2d, which is responsible to perform all the tasks in the backend.
* '''offscreenrenderingcontext2d.rs:''' : This file contains the implementation for OffscreenCanvasRenderingContext2D. Since it does not interact directly with JavaScript, neither does it have a constructor, nor does it have a reflector object. The functionalities mentioned in the interface for this file have to be implemented here. But that is beyond the scope of our current project. As of now, this file only contains the code to help successfully build an offscreencanvas object. The functionalities of this file are likely to be implemented in our subsequent project.

== Implementation ==

=== NOTE ===
The elements of the code added here has been explained in detail in the section: What we did -> under the hood.

=== OffScreenCanvas.webidl ===

<pre>
typedef (OffscreenCanvasRenderingContext2D or WebGLRenderingContext or WebGL2RenderingContext) OffscreenRenderingContext;

dictionary ImageEncodeOptions {
DOMString type = "image/png";
unrestricted double quality = 1.0;
};

enum OffscreenRenderingContextId { "2d", "webgl", "webgl2" };

[Constructor([EnforceRange] unsigned long long width, [EnforceRange] unsigned long long height), Exposed=(Window,Worker), Transferable]
interface OffscreenCanvas : EventTarget {
attribute [EnforceRange] unsigned long long width;
attribute [EnforceRange] unsigned long long height;

OffscreenRenderingContext? getContext(OffscreenRenderingContextId contextId, optional any options = null);
ImageBitmap transferToImageBitmap();
Promise<Blob> convertToBlob(optional ImageEncodeOptions options);
};
</pre>

=== OffScreenCanvasRenderingContext2D.webidl ===

<pre>

[Exposed=Window]
interface CanvasRenderingContext2D {
// back-reference to the canvas
readonly attribute HTMLCanvasElement canvas;
};
CanvasRenderingContext2D implements CanvasState;
CanvasRenderingContext2D implements CanvasTransform;
CanvasRenderingContext2D implements CanvasCompositing;
CanvasRenderingContext2D implements CanvasImageSmoothing;
CanvasRenderingContext2D implements CanvasFillStrokeStyles;
CanvasRenderingContext2D implements CanvasShadowStyles;
CanvasRenderingContext2D implements CanvasFilters;
CanvasRenderingContext2D implements CanvasRect;
CanvasRenderingContext2D implements CanvasDrawPath;
CanvasRenderingContext2D implements CanvasUserInterface;
CanvasRenderingContext2D implements CanvasText;
CanvasRenderingContext2D implements CanvasDrawImage;
CanvasRenderingContext2D implements CanvasImageData;
CanvasRenderingContext2D implements CanvasPathDrawingStyles;
CanvasRenderingContext2D implements CanvasTextDrawingStyles;
CanvasRenderingContext2D implements CanvasPath;

</pre>

=== offscreencanvasrenderingcontext2d.rs ===

<pre>

#[dom_struct]
pub struct OffscreenCanvasRenderingContext2D{
reflector_: Reflector,
canvas: Option<Dom<OffscreenCanvas>>,
}

impl OffscreenCanvasRenderingContext2D {
pub fn new_inherited(canvas: Option<&OffscreenCanvas>) -> OffscreenCanvasRenderingContext2D {
OffscreenCanvasRenderingContext2D {
reflector_: Reflector::new(),
canvas: canvas.map(Dom::from_ref),
}
}

pub fn new(canvas: Option<&OffscreenCanvas>) -> DomRoot<OffscreenCanvasRenderingContext2D> {
reflect_dom_object(Box::new(OffscreenCanvasRenderingContext2D::new_inherited(canvas)), OffscreenCanvasRenderingContext2DWrap)
}
}

</pre>

=== offscreencanvas.rs ===

<pre>

pub enum OffscreenRenderingContext {
Context2D(Dom<OffscreenCanvasRenderingContext2D>),
WebGL(Dom<WebGLRenderingContext>),
WebGL2(Dom<WebGL2RenderingContext>),
}

#[dom_struct]
pub struct OffscreenCanvas{
reflector_: Reflector,
height: u64,
width: u64,
context: DomRefCell<Option<OffscreenRenderingContext>>,
placeholder: Option<Dom<HTMLCanvasElement>>,
}

impl OffscreenCanvas{
pub fn new_inherited(height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> OffscreenCanvas {
OffscreenCanvas {
reflector_: Reflector::new(),
height: height,
width: width,
context: DomRefCell::new(None),
placeholder: placeholder.map(Dom::from_ref),
}
}

pub fn new(global: &GlobalScope, height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> DomRoot<OffscreenCanvas> {
reflect_dom_object(Box::new(OffscreenCanvas::new_inherited(height,width,placeholder)), global, OffscreenCanvasWrap)
}

pub fn Constructor (global: &GlobalScope, height: u64, width: u64) -> Fallible<DomRoot<OffscreenCanvas>> {
//step 1
let offscreencanvas = OffscreenCanvas::new(global,height,width,None);
//step 2

if offscreencanvas.context.borrow().is_some() {
return Err(Error::InvalidState);
}

offscreencanvas.height = height;
offscreencanvas.width = width;

offscreencanvas.placeholder = None;

//step 3
Ok(offscreencanvas)
}

pub fn context(&self) -> Option<Ref<OffscreenRenderingContext>> {
ref_filter_map::ref_filter_map(self.context.borrow(), |ctx| ctx.as_ref())
}

fn get_or_init_2d_context(&self) -> Option<DomRoot<OffscreenCanvasRenderingContext2D>> {
if let Some(ctx) = self.context() {
return match *ctx {
OffscreenRenderingContext::Context2D(ref ctx) => Some(DomRoot::from_ref(ctx)),
_ => None,
};
}
//let window = window_from_node(self);
//let size = self.get_size();
let context = OffscreenCanvasRenderingContext2D::new(self);
*self.context.borrow_mut() = Some(OffscreenRenderingContext::Context2D(Dom::from_ref(&*context)));
Some(context)
}

}

impl OffscreenCanvasMethods for OffscreenCanvas{
#[allow(unsafe_code)]
unsafe fn GetContext(&self,cx: *mut JSContext, contextID: DOMString, options: HandleValue) -> Option<OffscreenCanvasRenderingContext2DOrWebGLRenderingContextOrWebGL2RenderingContext> {

if contextID == "2d"
{
self.get_or_init_2d_context();
}
}

fn Width(&self) -> u64 {
return self.width;
}
fn SetWidth(&self, value: u64) -> () {
self.width = value;
}
fn Height(&self) -> u64 {
return self.height;
}
fn SetHeight(&self, value: u64) -> () {
self.height = value;
}
}

</pre>

== Test Plan ==

The changes we introduced could be tested using two ways.

* The first and recommended approach is to enable the existing automated tests for the OffScreenCanvas feature. We have enabled this by the following steps: 1. Added the offscreen-canvas directory to tests/wpt/include.ini 2. Added a __dir__.ini file to tests/wpt/metadata/offscreen-canvas which enables the new preference This, followed by updating the expected test results gave us the desired results.

* The second approach can be by writing your own test-cases. A sample HTML code to successfully test a rendered OffscreenCanvas is given below:

<pre>
<canvas id="testCanvas"></canvas>
<script>
var testCanvas = document.getElementById("testCanvas").getContext("bitmaprenderer");
var offscreen = new OffscreenCanvas(256,256);
var gl = offscreen.getContext('2d');

// any custom commands to draw something on the canvas (or leave it as it it for the sake of testing)

var bitmapOne = offscreen.transferToImageBitmap();
testCanvas.transferImageBitmap(bitmapOne);
</script>
</pre>
We have followed the first procedure where we plan to test the rendering of OffScreen canvas with the help of existing test cases already provided to us.

=== Pull Request ===

The pull request used to incorporate our changes upstream is available here [https://github.com/servo/servo/pull/22168].

== References: ==

1. https://research.mozilla.org/servo-engines/ 
2. https://research.mozilla.org/rust/ 
3. https://hacks.mozilla.org/2017/05/quantum-up-close-what-is-a-browser-engine/

CSC517 OffScreenCanvas Servo

2018-11-11T01:32:48Z

Psingh25: /* Overview */

== About Servo: ==

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except for the fact that it is specially tuned for better memory safety and concurrency features.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders, and constellation. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affect other tabs, browser or even other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

''Why use rust ?''

Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

== Before you get started ==
=== Off Screen Canvas: ===

Canvas is a popular way of drawing all kinds of graphics on the screen and an entry point to the world of WebGL. It can be used to draw shapes, images, run animations, or even display and process video content. It is often used to create beautiful user experiences in media-rich web applications and online games. Because canvas logic and rendering happens on the same thread as user interaction, the (sometimes heavy) computations involved in animations can harm the app’s real and perceived performance.

Until now, canvas drawing capabilities were tied to the <canvas> element, which meant it was directly depending on the DOM. OffscreenCanvas, as the name suggests, decouples the DOM and the Canvas API by moving it off-screen. Due to this decoupling, rendering of OffscreenCanvas is fully detached from the DOM and therefore offers some speed improvements over the regular canvas as there is no synchronization between the two.

===Web workers and Offscreen Canvas===

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages.
''But, how is OffscreenCanvas rendered using web-workers?:'' Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

=== Web IDL ===
Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an [[IDL specification language|IDL]] variant with:

* A number of features that allow one to more easily describe the behavior of common script objects in a web context.

* A mapping of how interfaces described with Web IDL correspond to language constructs within an [[ECMAScript]] execution environment.

== Scope: ==

The scope of the project included:

* Creating the OffscreenCanvas and OffscreenCanvasRenderingContext2d interfaces with stub method implementations.

* Implementing the OffscreenCanvas Constructor [https://html.spec.whatwg.org/multipage/scripting.html#dom-offscreencanvas] that creates a new canvas.

== Getting Started ==

=== Building Servo ===
* Testing servo requires it to build it locally, or using a cloud service like [http://janitor.technology Janitor]. 
* Building servo locally would take approximately 30mins - 1 hour depending upon your computer. 
* For further steps regarding building servo locally, please visit [https://github.com/servo/servo this]. 

=== Verifying Build ===
* We can verify whether servo has been built by running the following command: ./mach run http://www.google.com 
* This command will render the homepage of google in a new browser instance of servo. If this executes correctly, then the build is fine. 
* Specific functionalities could be tested by writing a custom html file and then running it the similar way. 
* Note: the following command won't work if you're on Janitor.

== What we did ==

=== Overview ===
* Our task was to implement the functionality of OffscreenCanvas. 
* Now, since OffscreenCanvas objects are declared and used in JavaScript, but the rendering happens outside the domain of JavaScript, we would need two different objects, one to interact with JavaScript, and the other to handle the rendering, independent of JavaScript. 
* The object to interact with JavaScript is OffscreenCanvas, whereas the object which handles rendering is OffscreenCanvasRenderingContext2D. 
* To implement a functionality, first we created the webIDL files, which contains interfaces of the functionalities we need to implement. 
* After creating the interfaces, the next step was to implement them. The implementation of the webIDL files is written in the rust file (.rs) with the same name. (For example, OffscreenCanvas.webidl -> offscreencanvas.rs) 
* The webIDL files contains the constructor as well as all the methods which were supposed to be implemented as specified in the webIDL files. 
* Testing the implemented code is discussed in further sections.

=== Code Flow ===
[[File:Workflow2108.png]]

=== Under the hood ===
* '''OffscreenCanvas.webidl:''' This is an interface, which defines what the implementation of the OffscreenCanvas should contain. As it directly interacts with JavaScript, hence it should have a constructor, with two arguments height and width, just like a regular HTMLCanvas element. The code for the same is shown in the implementation. Also, it contains the ''getContext'' method, which specifies the type of context: 2d, webgl or webgl2. 
* '''OffscreenCanvasRenderingContext2D.webidl:''' This is also an interface file, which defines what the implementation of OffscreenCanvasRenderingContext2D should look like. As it does not interacts directly with javascript, but interacts with the OffscreenCanvas object we created previously, it contains a back reference to the same. It also defines the functionalities which its implementation class should follow.
* '''offscreencanvas.rs:''' This file is responsible for implementing OffscreenCanvas.webidl. It must have a constructor, which is implemented as per the requirements of the webidl, posted in the section below (implementation). When a new instance of offscreencanvas is created, it calls the constructor, which in turn calls the new method, which ultimately calls the new method defined in this file. The arguments of the constructor, namely height, width and (weak reference) options, are stored in the new object which is created. It also contains a ''reflector'' object, which tells the JavaScript garbage collector to ignore the canvas object while clearing memory. It also has the implementation for the method ''getcontext()'', which is returns a weak reference to the OffscreenCanvasRenderingContext2d, which is responsible to perform all the tasks in the backend.
* '''offscreenrenderingcontext2d.rs:''' : This file contains the implementation for OffscreenCanvasRenderingContext2D. Since it does not interact directly with JavaScript, neither does it has a constructor, nor does it have a reflector object. The various functionalities mentioned in the interface are to be implemented with this file. But as of now, this file only contains the code to help successfully build an offscreencanvas object. The functionalities of this file would be implemented in the next project.

== Implementation ==

=== NOTE ===
The elements of the code added here has been explained in detail in the section: What we did -> under the hood.

=== OffScreenCanvas.webidl ===

<pre>
typedef (OffscreenCanvasRenderingContext2D or WebGLRenderingContext or WebGL2RenderingContext) OffscreenRenderingContext;

dictionary ImageEncodeOptions {
DOMString type = "image/png";
unrestricted double quality = 1.0;
};

enum OffscreenRenderingContextId { "2d", "webgl", "webgl2" };

[Constructor([EnforceRange] unsigned long long width, [EnforceRange] unsigned long long height), Exposed=(Window,Worker), Transferable]
interface OffscreenCanvas : EventTarget {
attribute [EnforceRange] unsigned long long width;
attribute [EnforceRange] unsigned long long height;

OffscreenRenderingContext? getContext(OffscreenRenderingContextId contextId, optional any options = null);
ImageBitmap transferToImageBitmap();
Promise<Blob> convertToBlob(optional ImageEncodeOptions options);
};
</pre>

=== OffScreenCanvasRenderingContext2D.webidl ===

<pre>

[Exposed=Window]
interface CanvasRenderingContext2D {
// back-reference to the canvas
readonly attribute HTMLCanvasElement canvas;
};
CanvasRenderingContext2D implements CanvasState;
CanvasRenderingContext2D implements CanvasTransform;
CanvasRenderingContext2D implements CanvasCompositing;
CanvasRenderingContext2D implements CanvasImageSmoothing;
CanvasRenderingContext2D implements CanvasFillStrokeStyles;
CanvasRenderingContext2D implements CanvasShadowStyles;
CanvasRenderingContext2D implements CanvasFilters;
CanvasRenderingContext2D implements CanvasRect;
CanvasRenderingContext2D implements CanvasDrawPath;
CanvasRenderingContext2D implements CanvasUserInterface;
CanvasRenderingContext2D implements CanvasText;
CanvasRenderingContext2D implements CanvasDrawImage;
CanvasRenderingContext2D implements CanvasImageData;
CanvasRenderingContext2D implements CanvasPathDrawingStyles;
CanvasRenderingContext2D implements CanvasTextDrawingStyles;
CanvasRenderingContext2D implements CanvasPath;

</pre>

=== offscreencanvasrenderingcontext2d.rs ===

<pre>

#[dom_struct]
pub struct OffscreenCanvasRenderingContext2D{
reflector_: Reflector,
canvas: Option<Dom<OffscreenCanvas>>,
}

impl OffscreenCanvasRenderingContext2D {
pub fn new_inherited(canvas: Option<&OffscreenCanvas>) -> OffscreenCanvasRenderingContext2D {
OffscreenCanvasRenderingContext2D {
reflector_: Reflector::new(),
canvas: canvas.map(Dom::from_ref),
}
}

pub fn new(canvas: Option<&OffscreenCanvas>) -> DomRoot<OffscreenCanvasRenderingContext2D> {
reflect_dom_object(Box::new(OffscreenCanvasRenderingContext2D::new_inherited(canvas)), OffscreenCanvasRenderingContext2DWrap)
}
}

</pre>

=== offscreencanvas.rs ===

<pre>

pub enum OffscreenRenderingContext {
Context2D(Dom<OffscreenCanvasRenderingContext2D>),
WebGL(Dom<WebGLRenderingContext>),
WebGL2(Dom<WebGL2RenderingContext>),
}

#[dom_struct]
pub struct OffscreenCanvas{
reflector_: Reflector,
height: u64,
width: u64,
context: DomRefCell<Option<OffscreenRenderingContext>>,
placeholder: Option<Dom<HTMLCanvasElement>>,
}

impl OffscreenCanvas{
pub fn new_inherited(height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> OffscreenCanvas {
OffscreenCanvas {
reflector_: Reflector::new(),
height: height,
width: width,
context: DomRefCell::new(None),
placeholder: placeholder.map(Dom::from_ref),
}
}

pub fn new(global: &GlobalScope, height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> DomRoot<OffscreenCanvas> {
reflect_dom_object(Box::new(OffscreenCanvas::new_inherited(height,width,placeholder)), global, OffscreenCanvasWrap)
}

pub fn Constructor (global: &GlobalScope, height: u64, width: u64) -> Fallible<DomRoot<OffscreenCanvas>> {
//step 1
let offscreencanvas = OffscreenCanvas::new(global,height,width,None);
//step 2

if offscreencanvas.context.borrow().is_some() {
return Err(Error::InvalidState);
}

offscreencanvas.height = height;
offscreencanvas.width = width;

offscreencanvas.placeholder = None;

//step 3
Ok(offscreencanvas)
}

pub fn context(&self) -> Option<Ref<OffscreenRenderingContext>> {
ref_filter_map::ref_filter_map(self.context.borrow(), |ctx| ctx.as_ref())
}

fn get_or_init_2d_context(&self) -> Option<DomRoot<OffscreenCanvasRenderingContext2D>> {
if let Some(ctx) = self.context() {
return match *ctx {
OffscreenRenderingContext::Context2D(ref ctx) => Some(DomRoot::from_ref(ctx)),
_ => None,
};
}
//let window = window_from_node(self);
//let size = self.get_size();
let context = OffscreenCanvasRenderingContext2D::new(self);
*self.context.borrow_mut() = Some(OffscreenRenderingContext::Context2D(Dom::from_ref(&*context)));
Some(context)
}

}

impl OffscreenCanvasMethods for OffscreenCanvas{
#[allow(unsafe_code)]
unsafe fn GetContext(&self,cx: *mut JSContext, contextID: DOMString, options: HandleValue) -> Option<OffscreenCanvasRenderingContext2DOrWebGLRenderingContextOrWebGL2RenderingContext> {

if contextID == "2d"
{
self.get_or_init_2d_context();
}
}

fn Width(&self) -> u64 {
return self.width;
}
fn SetWidth(&self, value: u64) -> () {
self.width = value;
}
fn Height(&self) -> u64 {
return self.height;
}
fn SetHeight(&self, value: u64) -> () {
self.height = value;
}
}

</pre>

== Test Plan ==

The changes we introduced could be tested using two ways.

* The first and recommended approach is to enable the existing automated tests for the OffScreenCanvas feature. We have enabled this by the following steps: 1. Added the offscreen-canvas directory to tests/wpt/include.ini 2. Added a __dir__.ini file to tests/wpt/metadata/offscreen-canvas which enables the new preference This, followed by updating the expected test results gave us the desired results.

* The second approach can be by writing your own test-cases. A sample HTML code to successfully test a rendered OffscreenCanvas is given below:

<pre>
<canvas id="testCanvas"></canvas>
<script>
var testCanvas = document.getElementById("testCanvas").getContext("bitmaprenderer");
var offscreen = new OffscreenCanvas(256,256);
var gl = offscreen.getContext('2d');

// any custom commands to draw something on the canvas (or leave it as it it for the sake of testing)

var bitmapOne = offscreen.transferToImageBitmap();
testCanvas.transferImageBitmap(bitmapOne);
</script>
</pre>
We have followed the first procedure where we plan to test the rendering of OffScreen canvas with the help of existing test cases already provided to us.

=== Pull Request ===

The pull request used to incorporate our changes upstream is available here [https://github.com/servo/servo/pull/22168].

== References: ==

1. https://research.mozilla.org/servo-engines/ 
2. https://research.mozilla.org/rust/ 
3. https://hacks.mozilla.org/2017/05/quantum-up-close-what-is-a-browser-engine/

CSC517 OffScreenCanvas Servo

2018-11-11T01:30:15Z

Psingh25: /* Test Plan */

== About Servo: ==

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except for the fact that it is specially tuned for better memory safety and concurrency features.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders, and constellation. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affect other tabs, browser or even other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

''Why use rust ?''

Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

== Before you get started ==
=== Off Screen Canvas: ===

Canvas is a popular way of drawing all kinds of graphics on the screen and an entry point to the world of WebGL. It can be used to draw shapes, images, run animations, or even display and process video content. It is often used to create beautiful user experiences in media-rich web applications and online games. Because canvas logic and rendering happens on the same thread as user interaction, the (sometimes heavy) computations involved in animations can harm the app’s real and perceived performance.

Until now, canvas drawing capabilities were tied to the <canvas> element, which meant it was directly depending on the DOM. OffscreenCanvas, as the name suggests, decouples the DOM and the Canvas API by moving it off-screen. Due to this decoupling, rendering of OffscreenCanvas is fully detached from the DOM and therefore offers some speed improvements over the regular canvas as there is no synchronization between the two.

===Web workers and Offscreen Canvas===

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages.
''But, how is OffscreenCanvas rendered using web-workers?:'' Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

=== Web IDL ===
Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an [[IDL specification language|IDL]] variant with:

* A number of features that allow one to more easily describe the behavior of common script objects in a web context.

* A mapping of how interfaces described with Web IDL correspond to language constructs within an [[ECMAScript]] execution environment.

== Scope: ==

The scope of the project included:

* Creating the OffscreenCanvas and OffscreenCanvasRenderingContext2d interfaces with stub method implementations.

* Implementing the OffscreenCanvas Constructor [https://html.spec.whatwg.org/multipage/scripting.html#dom-offscreencanvas] that creates a new canvas.

== Getting Started ==

=== Building Servo ===
* Testing servo requires it to build it locally, or using a cloud service like [http://janitor.technology Janitor]. 
* Building servo locally would take approximately 30mins - 1 hour depending upon your computer. 
* For further steps regarding building servo locally, please visit [https://github.com/servo/servo this]. 

=== Verifying Build ===
* We can verify whether servo has been built by running the following command: ./mach run http://www.google.com 
* This command will render the homepage of google in a new browser instance of servo. If this executes correctly, then the build is fine. 
* Specific functionalities could be tested by writing a custom html file and then running it the similar way. 
* Note: the following command won't work if you're on Janitor.

== What we did ==

=== Overview ===
* Our task was to implement the functionality of OffscreenCanvas. 
* Now, since OffscreenCanvas objects are declared and used in JavaScript, but the rendering happens outside the domain of JavaScript, we would need two different objects, one to interact with JavaScript, and the other to handle the rendering, independent of JavaScript. 
* The object to interact with JavaScript is OffscreenCanvas, whereas the object which handles rendering is OffscreenCanvasRenderingContext2D. 
* To implement a functionality, first we made the 'web-idl' files, which contains interfaces of the functionalities we need to implement. 
* After making the interfaces, the next step is to implement them. The implementation of the web-idl files are written in the rust file (.rs) with the same name. (For example, OffscreenCanvas.webidl -> offscreencanvas.rs) 
* The webidl files contains the constructor as well as all the methods which were supposed to be implemented as specified in the webidl files. 
* Testing the implemented code is discussed in further sections.

=== Code Flow ===
[[File:Workflow2108.png]]

=== Under the hood ===
* '''OffscreenCanvas.webidl:''' This is an interface, which defines what the implementation of the OffscreenCanvas should contain. As it directly interacts with JavaScript, hence it should have a constructor, with two arguments height and width, just like a regular HTMLCanvas element. The code for the same is shown in the implementation. Also, it contains the ''getContext'' method, which specifies the type of context: 2d, webgl or webgl2. 
* '''OffscreenCanvasRenderingContext2D.webidl:''' This is also an interface file, which defines what the implementation of OffscreenCanvasRenderingContext2D should look like. As it does not interacts directly with javascript, but interacts with the OffscreenCanvas object we created previously, it contains a back reference to the same. It also defines the functionalities which its implementation class should follow.
* '''offscreencanvas.rs:''' This file is responsible for implementing OffscreenCanvas.webidl. It must have a constructor, which is implemented as per the requirements of the webidl, posted in the section below (implementation). When a new instance of offscreencanvas is created, it calls the constructor, which in turn calls the new method, which ultimately calls the new method defined in this file. The arguments of the constructor, namely height, width and (weak reference) options, are stored in the new object which is created. It also contains a ''reflector'' object, which tells the JavaScript garbage collector to ignore the canvas object while clearing memory. It also has the implementation for the method ''getcontext()'', which is returns a weak reference to the OffscreenCanvasRenderingContext2d, which is responsible to perform all the tasks in the backend.
* '''offscreenrenderingcontext2d.rs:''' : This file contains the implementation for OffscreenCanvasRenderingContext2D. Since it does not interact directly with JavaScript, neither does it has a constructor, nor does it have a reflector object. The various functionalities mentioned in the interface are to be implemented with this file. But as of now, this file only contains the code to help successfully build an offscreencanvas object. The functionalities of this file would be implemented in the next project.

== Implementation ==

=== NOTE ===
The elements of the code added here has been explained in detail in the section: What we did -> under the hood.

=== OffScreenCanvas.webidl ===

<pre>
typedef (OffscreenCanvasRenderingContext2D or WebGLRenderingContext or WebGL2RenderingContext) OffscreenRenderingContext;

dictionary ImageEncodeOptions {
DOMString type = "image/png";
unrestricted double quality = 1.0;
};

enum OffscreenRenderingContextId { "2d", "webgl", "webgl2" };

[Constructor([EnforceRange] unsigned long long width, [EnforceRange] unsigned long long height), Exposed=(Window,Worker), Transferable]
interface OffscreenCanvas : EventTarget {
attribute [EnforceRange] unsigned long long width;
attribute [EnforceRange] unsigned long long height;

OffscreenRenderingContext? getContext(OffscreenRenderingContextId contextId, optional any options = null);
ImageBitmap transferToImageBitmap();
Promise<Blob> convertToBlob(optional ImageEncodeOptions options);
};
</pre>

=== OffScreenCanvasRenderingContext2D.webidl ===

<pre>

[Exposed=Window]
interface CanvasRenderingContext2D {
// back-reference to the canvas
readonly attribute HTMLCanvasElement canvas;
};
CanvasRenderingContext2D implements CanvasState;
CanvasRenderingContext2D implements CanvasTransform;
CanvasRenderingContext2D implements CanvasCompositing;
CanvasRenderingContext2D implements CanvasImageSmoothing;
CanvasRenderingContext2D implements CanvasFillStrokeStyles;
CanvasRenderingContext2D implements CanvasShadowStyles;
CanvasRenderingContext2D implements CanvasFilters;
CanvasRenderingContext2D implements CanvasRect;
CanvasRenderingContext2D implements CanvasDrawPath;
CanvasRenderingContext2D implements CanvasUserInterface;
CanvasRenderingContext2D implements CanvasText;
CanvasRenderingContext2D implements CanvasDrawImage;
CanvasRenderingContext2D implements CanvasImageData;
CanvasRenderingContext2D implements CanvasPathDrawingStyles;
CanvasRenderingContext2D implements CanvasTextDrawingStyles;
CanvasRenderingContext2D implements CanvasPath;

</pre>

=== offscreencanvasrenderingcontext2d.rs ===

<pre>

#[dom_struct]
pub struct OffscreenCanvasRenderingContext2D{
reflector_: Reflector,
canvas: Option<Dom<OffscreenCanvas>>,
}

impl OffscreenCanvasRenderingContext2D {
pub fn new_inherited(canvas: Option<&OffscreenCanvas>) -> OffscreenCanvasRenderingContext2D {
OffscreenCanvasRenderingContext2D {
reflector_: Reflector::new(),
canvas: canvas.map(Dom::from_ref),
}
}

pub fn new(canvas: Option<&OffscreenCanvas>) -> DomRoot<OffscreenCanvasRenderingContext2D> {
reflect_dom_object(Box::new(OffscreenCanvasRenderingContext2D::new_inherited(canvas)), OffscreenCanvasRenderingContext2DWrap)
}
}

</pre>

=== offscreencanvas.rs ===

<pre>

pub enum OffscreenRenderingContext {
Context2D(Dom<OffscreenCanvasRenderingContext2D>),
WebGL(Dom<WebGLRenderingContext>),
WebGL2(Dom<WebGL2RenderingContext>),
}

#[dom_struct]
pub struct OffscreenCanvas{
reflector_: Reflector,
height: u64,
width: u64,
context: DomRefCell<Option<OffscreenRenderingContext>>,
placeholder: Option<Dom<HTMLCanvasElement>>,
}

impl OffscreenCanvas{
pub fn new_inherited(height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> OffscreenCanvas {
OffscreenCanvas {
reflector_: Reflector::new(),
height: height,
width: width,
context: DomRefCell::new(None),
placeholder: placeholder.map(Dom::from_ref),
}
}

pub fn new(global: &GlobalScope, height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> DomRoot<OffscreenCanvas> {
reflect_dom_object(Box::new(OffscreenCanvas::new_inherited(height,width,placeholder)), global, OffscreenCanvasWrap)
}

pub fn Constructor (global: &GlobalScope, height: u64, width: u64) -> Fallible<DomRoot<OffscreenCanvas>> {
//step 1
let offscreencanvas = OffscreenCanvas::new(global,height,width,None);
//step 2

if offscreencanvas.context.borrow().is_some() {
return Err(Error::InvalidState);
}

offscreencanvas.height = height;
offscreencanvas.width = width;

offscreencanvas.placeholder = None;

//step 3
Ok(offscreencanvas)
}

pub fn context(&self) -> Option<Ref<OffscreenRenderingContext>> {
ref_filter_map::ref_filter_map(self.context.borrow(), |ctx| ctx.as_ref())
}

fn get_or_init_2d_context(&self) -> Option<DomRoot<OffscreenCanvasRenderingContext2D>> {
if let Some(ctx) = self.context() {
return match *ctx {
OffscreenRenderingContext::Context2D(ref ctx) => Some(DomRoot::from_ref(ctx)),
_ => None,
};
}
//let window = window_from_node(self);
//let size = self.get_size();
let context = OffscreenCanvasRenderingContext2D::new(self);
*self.context.borrow_mut() = Some(OffscreenRenderingContext::Context2D(Dom::from_ref(&*context)));
Some(context)
}

}

impl OffscreenCanvasMethods for OffscreenCanvas{
#[allow(unsafe_code)]
unsafe fn GetContext(&self,cx: *mut JSContext, contextID: DOMString, options: HandleValue) -> Option<OffscreenCanvasRenderingContext2DOrWebGLRenderingContextOrWebGL2RenderingContext> {

if contextID == "2d"
{
self.get_or_init_2d_context();
}
}

fn Width(&self) -> u64 {
return self.width;
}
fn SetWidth(&self, value: u64) -> () {
self.width = value;
}
fn Height(&self) -> u64 {
return self.height;
}
fn SetHeight(&self, value: u64) -> () {
self.height = value;
}
}

</pre>

== Test Plan ==

The changes we introduced could be tested using two ways.

* The first and recommended approach is to enable the existing automated tests for the OffScreenCanvas feature. We have enabled this by the following steps: 1. Added the offscreen-canvas directory to tests/wpt/include.ini 2. Added a __dir__.ini file to tests/wpt/metadata/offscreen-canvas which enables the new preference This, followed by updating the expected test results gave us the desired results.

* The second approach can be by writing your own test-cases. A sample HTML code to successfully test a rendered OffscreenCanvas is given below:

<pre>
<canvas id="testCanvas"></canvas>
<script>
var testCanvas = document.getElementById("testCanvas").getContext("bitmaprenderer");
var offscreen = new OffscreenCanvas(256,256);
var gl = offscreen.getContext('2d');

// any custom commands to draw something on the canvas (or leave it as it it for the sake of testing)

var bitmapOne = offscreen.transferToImageBitmap();
testCanvas.transferImageBitmap(bitmapOne);
</script>
</pre>
We have followed the first procedure where we plan to test the rendering of OffScreen canvas with the help of existing test cases already provided to us.

=== Pull Request ===

The pull request used to incorporate our changes upstream is available here [https://github.com/servo/servo/pull/22168].

== References: ==

1. https://research.mozilla.org/servo-engines/ 
2. https://research.mozilla.org/rust/ 
3. https://hacks.mozilla.org/2017/05/quantum-up-close-what-is-a-browser-engine/

CSC517 OffScreenCanvas Servo

2018-11-11T01:30:05Z

Psingh25: /* Pull Request */

== About Servo: ==

Servo is a parallel web-engine project under Mozilla, which aims to develop a web engine to better utilize the potential of multiple processing units to parse and display web-pages faster than conventional browser engines. Servo is implemented using Rust programming language, which is similar to C, except for the fact that it is specially tuned for better memory safety and concurrency features.

Servo makes web-page rendering faster by using parallel layout, styling, web-renders, and constellation. Since, when a thread or javascript of a particular section of the webpage fails, it doesn't affect other tabs, browser or even other elements of the same web-page. So, in a way, Rust lets the browser to "fail better" than other browsers.

''Why use rust ?''

Algorithms which deal with multiple threads running in parallel are inherently difficult, due to synchronization, data sharing and other typical issues which arise while designing parallel algorithms. C++ makes this process even harder as it does not have efficient mechanisms to deal with this. So, the idea behind rust and servo is to rewrite C++ (and create rust) and use that to rewrite a browser.

== Before you get started ==
=== Off Screen Canvas: ===

Canvas is a popular way of drawing all kinds of graphics on the screen and an entry point to the world of WebGL. It can be used to draw shapes, images, run animations, or even display and process video content. It is often used to create beautiful user experiences in media-rich web applications and online games. Because canvas logic and rendering happens on the same thread as user interaction, the (sometimes heavy) computations involved in animations can harm the app’s real and perceived performance.

Until now, canvas drawing capabilities were tied to the <canvas> element, which meant it was directly depending on the DOM. OffscreenCanvas, as the name suggests, decouples the DOM and the Canvas API by moving it off-screen. Due to this decoupling, rendering of OffscreenCanvas is fully detached from the DOM and therefore offers some speed improvements over the regular canvas as there is no synchronization between the two.

===Web workers and Offscreen Canvas===

Web Workers makes it possible to run a script operation in a background thread separate from the main execution thread of a web application. The advantage of this is that laborious processing can be performed in a separate thread, allowing the main (usually the UI) thread to run without being blocked/slowed down. So, JavaScript could run in the background without affecting the performance of the webpage, independent of other scripts. But, web-workers are treated as external files, which makes it impossible for them to access window object, document object and the parent object. Hence, web-workers can only communicate with the rendering-engine by passing messages.
''But, how is OffscreenCanvas rendered using web-workers?:'' Suppose the task is to render an image in the canvas. The rendering engine creates a web-worker, and sends it the image data that it needs to process. The web-worker does the required processing, and returns the resultant image back to the rendering-engine. In this way, an image is rendered using an entirely independent process from the webpage.

=== Web IDL ===
Web IDL is an interface definition language that can be used to describe interfaces that are intended to be implemented in web browsers. It is an [[IDL specification language|IDL]] variant with:

* A number of features that allow one to more easily describe the behavior of common script objects in a web context.

* A mapping of how interfaces described with Web IDL correspond to language constructs within an [[ECMAScript]] execution environment.

== Scope: ==

The scope of the project included:

* Creating the OffscreenCanvas and OffscreenCanvasRenderingContext2d interfaces with stub method implementations.

* Implementing the OffscreenCanvas Constructor [https://html.spec.whatwg.org/multipage/scripting.html#dom-offscreencanvas] that creates a new canvas.

== Getting Started ==

=== Building Servo ===
* Testing servo requires it to build it locally, or using a cloud service like [http://janitor.technology Janitor]. 
* Building servo locally would take approximately 30mins - 1 hour depending upon your computer. 
* For further steps regarding building servo locally, please visit [https://github.com/servo/servo this]. 

=== Verifying Build ===
* We can verify whether servo has been built by running the following command: ./mach run http://www.google.com 
* This command will render the homepage of google in a new browser instance of servo. If this executes correctly, then the build is fine. 
* Specific functionalities could be tested by writing a custom html file and then running it the similar way. 
* Note: the following command won't work if you're on Janitor.

== What we did ==

=== Overview ===
* Our task was to implement the functionality of OffscreenCanvas. 
* Now, since OffscreenCanvas objects are declared and used in JavaScript, but the rendering happens outside the domain of JavaScript, we would need two different objects, one to interact with JavaScript, and the other to handle the rendering, independent of JavaScript. 
* The object to interact with JavaScript is OffscreenCanvas, whereas the object which handles rendering is OffscreenCanvasRenderingContext2D. 
* To implement a functionality, first we made the 'web-idl' files, which contains interfaces of the functionalities we need to implement. 
* After making the interfaces, the next step is to implement them. The implementation of the web-idl files are written in the rust file (.rs) with the same name. (For example, OffscreenCanvas.webidl -> offscreencanvas.rs) 
* The webidl files contains the constructor as well as all the methods which were supposed to be implemented as specified in the webidl files. 
* Testing the implemented code is discussed in further sections.

=== Code Flow ===
[[File:Workflow2108.png]]

=== Under the hood ===
* '''OffscreenCanvas.webidl:''' This is an interface, which defines what the implementation of the OffscreenCanvas should contain. As it directly interacts with JavaScript, hence it should have a constructor, with two arguments height and width, just like a regular HTMLCanvas element. The code for the same is shown in the implementation. Also, it contains the ''getContext'' method, which specifies the type of context: 2d, webgl or webgl2. 
* '''OffscreenCanvasRenderingContext2D.webidl:''' This is also an interface file, which defines what the implementation of OffscreenCanvasRenderingContext2D should look like. As it does not interacts directly with javascript, but interacts with the OffscreenCanvas object we created previously, it contains a back reference to the same. It also defines the functionalities which its implementation class should follow.
* '''offscreencanvas.rs:''' This file is responsible for implementing OffscreenCanvas.webidl. It must have a constructor, which is implemented as per the requirements of the webidl, posted in the section below (implementation). When a new instance of offscreencanvas is created, it calls the constructor, which in turn calls the new method, which ultimately calls the new method defined in this file. The arguments of the constructor, namely height, width and (weak reference) options, are stored in the new object which is created. It also contains a ''reflector'' object, which tells the JavaScript garbage collector to ignore the canvas object while clearing memory. It also has the implementation for the method ''getcontext()'', which is returns a weak reference to the OffscreenCanvasRenderingContext2d, which is responsible to perform all the tasks in the backend.
* '''offscreenrenderingcontext2d.rs:''' : This file contains the implementation for OffscreenCanvasRenderingContext2D. Since it does not interact directly with JavaScript, neither does it has a constructor, nor does it have a reflector object. The various functionalities mentioned in the interface are to be implemented with this file. But as of now, this file only contains the code to help successfully build an offscreencanvas object. The functionalities of this file would be implemented in the next project.

== Test Plan ==

The changes we introduced could be tested using two ways.

* The first and recommended approach is to enable the existing automated tests for the OffScreenCanvas feature. We have enabled this by the following steps: 1. Added the offscreen-canvas directory to tests/wpt/include.ini 2. Added a __dir__.ini file to tests/wpt/metadata/offscreen-canvas which enables the new preference This, followed by updating the expected test results gave us the desired results.

* The second approach can be by writing your own test-cases. A sample HTML code to successfully test a rendered OffscreenCanvas is given below:

<pre>
<canvas id="testCanvas"></canvas>
<script>
var testCanvas = document.getElementById("testCanvas").getContext("bitmaprenderer");
var offscreen = new OffscreenCanvas(256,256);
var gl = offscreen.getContext('2d');

// any custom commands to draw something on the canvas (or leave it as it it for the sake of testing)

var bitmapOne = offscreen.transferToImageBitmap();
testCanvas.transferImageBitmap(bitmapOne);
</script>
</pre>
We have followed the first procedure where we plan to test the rendering of OffScreen canvas with the help of existing test cases already provided to us.

== Implementation ==

=== NOTE ===
The elements of the code added here has been explained in detail in the section: What we did -> under the hood.

=== OffScreenCanvas.webidl ===

<pre>
typedef (OffscreenCanvasRenderingContext2D or WebGLRenderingContext or WebGL2RenderingContext) OffscreenRenderingContext;

dictionary ImageEncodeOptions {
DOMString type = "image/png";
unrestricted double quality = 1.0;
};

enum OffscreenRenderingContextId { "2d", "webgl", "webgl2" };

[Constructor([EnforceRange] unsigned long long width, [EnforceRange] unsigned long long height), Exposed=(Window,Worker), Transferable]
interface OffscreenCanvas : EventTarget {
attribute [EnforceRange] unsigned long long width;
attribute [EnforceRange] unsigned long long height;

OffscreenRenderingContext? getContext(OffscreenRenderingContextId contextId, optional any options = null);
ImageBitmap transferToImageBitmap();
Promise<Blob> convertToBlob(optional ImageEncodeOptions options);
};
</pre>

=== OffScreenCanvasRenderingContext2D.webidl ===

<pre>

[Exposed=Window]
interface CanvasRenderingContext2D {
// back-reference to the canvas
readonly attribute HTMLCanvasElement canvas;
};
CanvasRenderingContext2D implements CanvasState;
CanvasRenderingContext2D implements CanvasTransform;
CanvasRenderingContext2D implements CanvasCompositing;
CanvasRenderingContext2D implements CanvasImageSmoothing;
CanvasRenderingContext2D implements CanvasFillStrokeStyles;
CanvasRenderingContext2D implements CanvasShadowStyles;
CanvasRenderingContext2D implements CanvasFilters;
CanvasRenderingContext2D implements CanvasRect;
CanvasRenderingContext2D implements CanvasDrawPath;
CanvasRenderingContext2D implements CanvasUserInterface;
CanvasRenderingContext2D implements CanvasText;
CanvasRenderingContext2D implements CanvasDrawImage;
CanvasRenderingContext2D implements CanvasImageData;
CanvasRenderingContext2D implements CanvasPathDrawingStyles;
CanvasRenderingContext2D implements CanvasTextDrawingStyles;
CanvasRenderingContext2D implements CanvasPath;

</pre>

=== offscreencanvasrenderingcontext2d.rs ===

<pre>

#[dom_struct]
pub struct OffscreenCanvasRenderingContext2D{
reflector_: Reflector,
canvas: Option<Dom<OffscreenCanvas>>,
}

impl OffscreenCanvasRenderingContext2D {
pub fn new_inherited(canvas: Option<&OffscreenCanvas>) -> OffscreenCanvasRenderingContext2D {
OffscreenCanvasRenderingContext2D {
reflector_: Reflector::new(),
canvas: canvas.map(Dom::from_ref),
}
}

pub fn new(canvas: Option<&OffscreenCanvas>) -> DomRoot<OffscreenCanvasRenderingContext2D> {
reflect_dom_object(Box::new(OffscreenCanvasRenderingContext2D::new_inherited(canvas)), OffscreenCanvasRenderingContext2DWrap)
}
}

</pre>

=== offscreencanvas.rs ===

<pre>

pub enum OffscreenRenderingContext {
Context2D(Dom<OffscreenCanvasRenderingContext2D>),
WebGL(Dom<WebGLRenderingContext>),
WebGL2(Dom<WebGL2RenderingContext>),
}

#[dom_struct]
pub struct OffscreenCanvas{
reflector_: Reflector,
height: u64,
width: u64,
context: DomRefCell<Option<OffscreenRenderingContext>>,
placeholder: Option<Dom<HTMLCanvasElement>>,
}

impl OffscreenCanvas{
pub fn new_inherited(height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> OffscreenCanvas {
OffscreenCanvas {
reflector_: Reflector::new(),
height: height,
width: width,
context: DomRefCell::new(None),
placeholder: placeholder.map(Dom::from_ref),
}
}

pub fn new(global: &GlobalScope, height: u64, width: u64, placeholder: Option<&HTMLCanvasElement>) -> DomRoot<OffscreenCanvas> {
reflect_dom_object(Box::new(OffscreenCanvas::new_inherited(height,width,placeholder)), global, OffscreenCanvasWrap)
}

pub fn Constructor (global: &GlobalScope, height: u64, width: u64) -> Fallible<DomRoot<OffscreenCanvas>> {
//step 1
let offscreencanvas = OffscreenCanvas::new(global,height,width,None);
//step 2

if offscreencanvas.context.borrow().is_some() {
return Err(Error::InvalidState);
}

offscreencanvas.height = height;
offscreencanvas.width = width;

offscreencanvas.placeholder = None;

//step 3
Ok(offscreencanvas)
}

pub fn context(&self) -> Option<Ref<OffscreenRenderingContext>> {
ref_filter_map::ref_filter_map(self.context.borrow(), |ctx| ctx.as_ref())
}

fn get_or_init_2d_context(&self) -> Option<DomRoot<OffscreenCanvasRenderingContext2D>> {
if let Some(ctx) = self.context() {
return match *ctx {
OffscreenRenderingContext::Context2D(ref ctx) => Some(DomRoot::from_ref(ctx)),
_ => None,
};
}
//let window = window_from_node(self);
//let size = self.get_size();
let context = OffscreenCanvasRenderingContext2D::new(self);
*self.context.borrow_mut() = Some(OffscreenRenderingContext::Context2D(Dom::from_ref(&*context)));
Some(context)
}

}

impl OffscreenCanvasMethods for OffscreenCanvas{
#[allow(unsafe_code)]
unsafe fn GetContext(&self,cx: *mut JSContext, contextID: DOMString, options: HandleValue) -> Option<OffscreenCanvasRenderingContext2DOrWebGLRenderingContextOrWebGL2RenderingContext> {

if contextID == "2d"
{
self.get_or_init_2d_context();
}
}

fn Width(&self) -> u64 {
return self.width;
}
fn SetWidth(&self, value: u64) -> () {
self.width = value;
}
fn Height(&self) -> u64 {
return self.height;
}
fn SetHeight(&self, value: u64) -> () {
self.height = value;
}
}

</pre>

== Test Plan ==

The changes we introduced could be tested using two ways.

* The first and recommended approach is to enable the existing automated tests for the OffScreenCanvas feature. We have enabled this by the following steps: 1. Added the offscreen-canvas directory to tests/wpt/include.ini 2. Added a __dir__.ini file to tests/wpt/metadata/offscreen-canvas which enables the new preference This, followed by updating the expected test results gave us the desired results.

* The second approach can be by writing your own test-cases. A sample HTML code to successfully test a rendered OffscreenCanvas is given below:

<pre>
<canvas id="testCanvas"></canvas>
<script>
var testCanvas = document.getElementById("testCanvas").getContext("bitmaprenderer");
var offscreen = new OffscreenCanvas(256,256);
var gl = offscreen.getContext('2d');

// any custom commands to draw something on the canvas (or leave it as it it for the sake of testing)

var bitmapOne = offscreen.transferToImageBitmap();
testCanvas.transferImageBitmap(bitmapOne);
</script>
</pre>
We have followed the first procedure where we plan to test the rendering of OffScreen canvas with the help of existing test cases already provided to us.

=== Pull Request ===

The pull request used to incorporate our changes upstream is available here [https://github.com/servo/servo/pull/22168].

== References: ==

1. https://research.mozilla.org/servo-engines/ 
2. https://research.mozilla.org/rust/ 
3. https://hacks.mozilla.org/2017/05/quantum-up-close-what-is-a-browser-engine/