PeerLogic Web Services

From Expertiza_Wiki
Jump to navigation Jump to search

Summary

The PeerLogic web services are a group of scripts and programs utilized by the expertiza website for performing various functions. These functions are hosted at peerlogic.csc.ncsu.edu and are externally called for the purposes of better performance, flexibility, and modularity.

This page is documenting all of these services, including where they are located, the code they utilize, and theoretical explanations of what they are performing.

The collection of the code for all of these web services can be found here.

Documentation Videos (Start Here!)

If you are just starting out accessing the PeerLogic Web Services, a series of videos have been created to explain the general premise of the web services, demonstrate some of the information expressed below, and more. This is a good way to get a general overview of how the Web Services work. To view these videos, click here.

Current Status of Web Services

The status of all currently supported or developed web services can be found at this google sheet (note: this link is editable, as this document should be updated as new services are added or modified. Please edit responsibly!): https://docs.google.com/spreadsheets/d/1oMYqjaVInPEE4jE7C2iQmOTc0H6BsltnuXXf0UWvuPA/edit?usp=sharing.

This sheet also notes the ports which are currently used by all services, and the URLs each service is accessible from.

Getting Started on Updating the Server

If for some reason, you need to modify, change, or update the code hosted at peerlogic.csc.ncsu.edu, you may follow these steps to do so.

  • Submit a request to CSC IT in order to be given access to the server. You can do this by emailing them at csc_help@ncsu.edu and cc'ing Dr. Gehringer so he may verify you have permission to do so.
  • Prepare the code you intend to add or modify in a GitHub repository. This will make it as simple as possible to transport your code to the web server. Ensure that your application is set up to run on a currently unoccupied port. The list of currently used ports can be found on the first sheet of the configuration sheet linked here.
  • Transfer that code over to the PeerLogic GitHub group (if you are not part of the group, join it!) This will ensure anyone in the future will be able to access and modify your code.
  • The server can only be accessed from within the internal NCState network. In order to gain this access, create a VCL reservation here, and SSH or remote desktop into the VCL. For the purposes of this explanation, I will assume you are doing this using Unix for your VCL.
  • Now that you are within the network, you can access the server via the SSH command, using your unityid. You will also be asked to provide your password upon connecting to the server. The command to ssh is given below:
ssh <unityid>@peerlogic.csc.ncsu.edu
  • Navigate to where the web services are located, using the following command:
cd /opt/webservices/
  • Located here are GitHub repos of all of the web services. If you intend to update an already-deployed service, simply enter the repository for that service and perform the following command to pull the changes:
git pull
  • Make sure the changes are working by running the new version locally and testing using API (see demonstration videos for more details)
  • Make sure the code still works the next day (after the server reset)

Adding a New Web Service

Template and Walkthrough

If you are just starting out and are willing to work in Python, a web service template has been created with explicit instructions, in both written and video form. It is recommended to use this as your approach unless you have something else specific in mind. To find these resources, click here: https://expertiza.csc.ncsu.edu/index.php/PeerLogic_Web_Services:_Python_Web_Service_Template.

General Instructions

  • First, clone your remote PeerLogic GitHub repository into the webservices directory under /opt/webservices. This will put the application code onto the server.
  • Next, you will need to edit one of the two startup scripts in /opt/webservices: runws_root.sh, and runws_user.sh.
  • runws_user.sh belongs to the "railsadmin" user and runs ruby 2.3.0p0. This file should be modified in order to start up any Ruby services.
  • runws_root.sh, meanwhile, is run by the root access user, and should be used for any Python scripts or anything needing root access.
  • Pick the appropriate script and add the code to start your application within (see the video demonstration for more details on this).
  • After doing all this, navigate to and edit /etc/nginx/nginx.conf (you will need sudo access to do this)
  • Within this config file, each service is assigned to a unique port on the machine. Your application must be configured to run on one of these ports, and on a port not already in use (please see the spreadsheet to see which ports are already in use).
  • To add your application to these url mappings, add this code to the config file, substituting the urls relevant to your service (this is very similar to the already existing mappings within the file):
location ^~ /[URL_PATH_TOBE_EXPOSED_EXTERNALLY] {
        	proxy_pass [YOUR_SERVICE_LOCAL_URL];
}
  • Reload nginx to update the changes to the config file and allow the new route to be properly redirected using the following:
sudo service nginx reload
  • After the next server restart, your service should be automatically started. You can test whether the new service is working immediately by running it yourself, and setting it to automatically run in the background. If you were running a python script, this may be something like the following:
sudo nohup python3 <your_application_name.py> &

NOTE: it is highly recommended to watch the demonstration video for setting up a new WebService, particularly if working in Python, which can be viewed at the page linked in the above section, as well as by clicking this link (while logged into your NCState account): https://drive.google.com/file/d/15Q5hBXU4umpJN8gStLryo5XdNiy1xZay/view?usp=sharing.

Currently Active Web Services

For the sake of this section, "currently active" means that these services are currently running on the PeerLogic web server and have known inputs, such that we can get useful results from them. Other web services may also technically be accessible, but may not currently have known inputs.

IntelligentAssignment

IntelligentAssignment is a Python application which has the stated purpose of forming project groups by performing k-means clustering on their topics of interest.

This service is currently being used by the Expertiza website for forming project groups.

Please see the extended documentation here.

This project was last updated in March of 2018.

The code for this project can be accessed here: https://github.com/peerlogic/IntelligentAssignment.

RainbowGraphService

RainbowGraphService is a primarily JavaScript application that visualizes the results of peer review criticisms.

The RainbowGraphService is actively used by collaborators outside the university for visualizations.

Please see the extended documentation here.

This project was last updated in March of 2018.

The code for this project can be accessed here: https://github.com/peerlogic/RainbowGraphService.

reputation

reputation is a Javascript application with extensive documentation, detailing the web service's use for determining user reputation.

This web service is not currently utilized, though students have recently (in Spring of 2022) created a project to utilize this service on Expertiza.

Please see the extended documentation for this project here: here.

This project was last updated in April of 2016.

The code for this project can be accessed here https://github.com/peerlogic/reputation_web_service.

Peer-reviews-NLP

Peer-reviews-NLP is a Python-based program which is used to tag student reviews. It does so by performing NLP and identifying features such as the presence of suggestions, problems, and more. Currently, 5 options are known to be functioning and useful: volume, suggestions, suggestions_confidence, problems, and problems_confidence.

This web service is actively utilized by Expertiza for automated tagging of reviews. It should be noted that this service is not run on the PeerLogic web server, but is instead run on the active learning web server, at http://152.7.99.200:5000. Please see the README in the GitHub or the documentation below for more information.

Please see the extended documentation here.

This project was last updated in March of 2018.

The code for this project can be accessed here: [1].

AllReviewInterface

AllReviewInterface is a Python-based service that allows one to send a collection of student reviews to several of the Peer-reviews-NLP services in one function call, returning the results from each.

This web service is not intended for particular use (apart from perhaps being convenient for certain research applications), and was primarily created to serve as a template for future web services.

This project does not have extended documentation, but the specifics of how to create a new web service using it as a template can be found here, and necessary information to run the service can be found in its GitHub README.

The code for this project can be accessed here: [2].

Other Documented Services

Besides the active web services listed above, several other services have existing documentation, despite their currently non-functional state. It is possible these services could be upgraded to "active" if one determined the input they required or fixed the deployment issues they may have.

AutoSummaryV1

AutoSummaryV1 is a Javascript application about explaining and summarizing reviews made by students.

Please see the extended documentation here: here.

This project was last updated in February of 2016.

The code for this project can be accessed here: https://github.com/peerlogic/AutoSummaryV1.

autometareviews0.1

This project is a Ruby application for the purpose of natural language processing, including sentiment analysis, volume, tone, etc.

This project was last updated in April of 2016.

The code for this project can be accessed here https://github.com/peerlogic/autometareviews0.1.

Legacy Resources

All prior documentation was originally created by Geoff Garrido in Spring of 2022, to replace and expand upon existing documentation.

The following are saved legacy resources of that prior documentation, in the case that the above documentation is insufficient or more nuanced information about how the PeerLogic server used to run is necessary.

Original PeerLogic web service set up: here. NLP Web Service Setup: here.

Documentation and Backup Drive

A Google Drive has been created containing all the videos, spreadsheets, presentations, and backups for the PeerLogic web services. It can be accessed here: https://drive.google.com/drive/folders/1yfCSXfu9NMUyKxIsvIYEQZkVUISVWtwB?usp=sharing.

In particular, because they are not backed up in any form of GitHub repositories (unlike the project code), this Google Drive contains backups of the most important configuration files for the PeerLogic web server: nginx.conf, runws_root.sh, runws_user.sh, and the contents of the cron jobs. Two versions of these files currently exist: the ones containing in their title "fall2021" refer to the state of the PeerLogic Web Services before the work done in Spring 2022 by Geoff Garrido. Those containing in their title "spring2022" refer to the state of the PeerLogic Web Services after the work done in Spring of 2022.

The more recent versions should be used as backups in all cases, unless something about the new versions is wrong, in which case the legacy versions are also provided.

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=PeerLogic_Web_Services&oldid=145702"