CSC/ECE 517 Spring 2025 - E2508. Reimplement bidding-algorithm web service

From Expertiza_Wiki
Jump to navigation Jump to search

Project Overview

Background

The previous implementation of the bidding service was hosted on peerlogic by a former student. The University no longer has access to this hosting service, and is requiring a new solution. It is also asked that the webservice accept a better and cleaner json format which is easier to interpret and read.

Objectives

  1. Reimplement trading cycles on a webservice
  2. Figure out a new hosting solution for the BiddingService web service. (VCL)
  3. Host the webservice on VCL
  4. Using tmux, figure out a way to keep the process running with a production wsgi service compatible with Flask.
  5. Create new tests for the repository
  6. Create a list of required packages that future maintainers can install easily
  7. Create a tests directory that users can test the functionality of the code with
  8. Modularize the code further, eliminate code smells.
  9. Refactor existing backend logic to account for the new json format
  10. Refactor existing backend code to send HTTP requests to the correct IP address
  11. Refactor existing frontend code to provide a list for users to select multiple topics on the frontend
  12. Allow users to prioritize their topic selection on the frontend
  13. Users should not be allowed to bid on their review topics on the frontend
  14. Display the results of the matched topics on the frontend

Previous Work

There have been three previous implementations:
Fall 2019 here
Spring 2020 here
Fall 2020 here
Fall 2024 here

Relevant Links

Github Repository for Expertiza: https://github.com/ncsu-ngw/expertiza/tree/e2508-reimplement-bidding-algorithm-web-service
Github Repository for Web service: https://github.com/ncsu-ngw/BiddingService
Pull Request: https://github.com/expertiza/expertiza/pull/2931
Web Service Server: 152.7.178.10:8080/match_topics

Files Modified

This is a list of all the files added or changed in completing this project.

Changed:

  • app/app.py - accomodated for changes in JsonParser
  • app/json_parser.py - moved functions out due to SRP
  • app/topics_matcher.py



Added:

  • app/tests/sample.json
    - Sample input for the tests.
  • app/tests/test_assignment.py
    Test File to test forms of input - tests both a successful and unsuccessful input
  • app/utils/popular_topics.py
    - Json Parser SRP violation - moved to own file
  • app/utils/student_priorities.py
    - Json Parser SRP violation - moved to own file
  • app/utils/topic_priorities.py
    - Json Parser SRP violation - moved to own file
  • setup.py - required for wsgi
  • wsgi.py - for hosting the server via gunicorn on VCL.
  • .gitignore - to ignore anything that we may not want pushed (mostly for venv)
  • requirements.txt - a list of requirements made easy for future maintainers

Webservice

app

The entry point to the Web service is the app.py file where we have the POST Method match_topics. This receives the input in a json format from Expertiza. The input will contain a list of task ids, and then an Object of users, which each contain their student id and correlated bids that they made. Each bid contains a task id, a priority, and a timestamp. The input format was reworked to align to this style since it made more sense to have each student have a list of bids instead of the previous implementation which did not have this case.



json parser

The json parser converts the json into a Dictionary. The previous implementation also had the parser create a preferences dictionary for the Student to store their topic preferences and one for Topics to store list of topics which have been selected by students and also the priority for the selection using the Timestamps. However, this was removed due to the Single Responsibility Principle. We felt the json parser was doing too much and should only be parsing the json!


The student priorities function in its own file, moved due to SRP.


The topic priorities function in its own file, moved due to SRP.



Test Input

Sample Test case:
Case: There are a few very popular topics and the students who bid for them and bid early should receive it. Popular Topics: 3969, 3971, 3972 

{
 {
   "tid": [4427, 4428, 4429, 4430],
   "users": {
     "40763": {
         "bids": [
         { "tid": 4430, "priority": 3, "timestamp": "Sun, 15 Nov 2020 17:16:34 EST -05:00" },
         { "tid": 4428, "priority": 2, "timestamp": "Sun, 15 Nov 2020 17:16:35 EST -05:00" },
         { "tid": 4427, "priority": 1, "timestamp": "Sun, 15 Nov 2020 17:16:37 EST -05:00" }
       ],
       "otid": 4429
     },
     "40764": {
       "bids": [
         { "tid": 4429, "priority": 3, "timestamp": "Sun, 15 Nov 2020 17:16:34 EST -05:00" },
         { "tid": 4430, "priority": 2, "timestamp": "Sun, 15 Nov 2020 17:16:35 EST -05:00" },
         { "tid": 4428, "priority": 1, "timestamp": "Sun, 15 Nov 2020 17:16:37 EST -05:00" }
       ],
       "otid": 4427
     },
     "40765": {
       "bids": [
         { "tid": 4427, "priority": 3, "timestamp": "Sun, 15 Nov 2020 17:17:15 EST -05:00" },
         { "tid": 4428, "priority": 1, "timestamp": "Sun, 15 Nov 2020 17:17:16 EST -05:00" },
         { "tid": 4429, "priority": 2, "timestamp": "Sun, 15 Nov 2020 17:17:17 EST -05:00" }
       ],
       "otid": 4428
     }
   },
   "max_accepted_proposals": 3
 }


Output
Since there is competition for these topics, the algorithm recognizes this and makes sure to assign them only to the students who bid for it.

Other students who picked other topics or did not pick anything are assigned different topics. 

{
   "40763": [
       4427,
       4428,
       4430
   ],
   "40764": [
       4428,
       4430,
       4429
   ],
   "40765": [
       4429,
       4427,
       4430
   ]
}

Controller

The controller includes 8 methods that handle the review bidding process. The public methods cover authorization, rendering views, saving user bids, and coordinating review assignment. The private methods support these functions by abstracting data gathering and algorithm coordination logic.


Method: action_allowed?

This method controls user access to controller actions. It allows 'Instructor', 'Teaching Assistant', 'Administrator', 'Super-Administrator', and 'Student' roles to access show, set_priority, and index. Other actions are restricted to non-student roles. It ensures proper permissions for accessing review bidding features.

Method: index

This method gathers information for the "Your Work" page, where users can view their assigned reviews. It checks user identity, retrieves associated reviews, and counts completed reviews. The view rendered is sign_up_sheet/review_bids_others_work.View: others_work.html.erb

]

Method: show

This method renders the bidding page where users select topics they'd prefer to review. It fetches the assignment, available topics, and any existing bids. Already assigned or signed-up topics are excluded. The page also lists any assigned reviews.View: show.html.erb

]

Method: set_priority

This method stores or updates a participant's topic preferences. It removes old bids not in the current selection and creates or updates the remaining bids with new priorities. This method is triggered when users submit their topic rankings on the bidding page.

]

Method: assign_bidding

This method coordinates the assignment of review topics. It gathers reviewer IDs and invokes the bidding algorithm using a service. It validates the output and uses ReviewBid.assign_review_topics to perform the actual assignment. Bidding is then disabled.

]

Method: fetch_reviewer_ids (Private)

This helper method retrieves all AssignmentParticipant IDs for a given assignment. These IDs represent the reviewers participating in the bidding process.

Method: process_bidding (Private)

This helper method delegates the logic for running the bidding algorithm to the service class ReviewBiddingAlgorithmService. It takes the assignment ID and list of reviewers as input and returns matched topic assignments.

Method: ensure_valid_topics (Private)

This method ensures that all reviewers have valid topic assignment entries, even if the algorithm returns nothing (e.g., due to web service failure). It initializes empty topic arrays for any missing reviewers.

]


Model

The model includes 13 methods that manage the review bidding process. The public methods handle bidding data retrieval, review topic assignment, and fallback allocation. The private methods support these functions by fetching user and team data, sorting topics, and formatting bid details.

Method: bidding_data

Collects bidding data for an assignment, including topics, max allowed proposals, and individual reviewer bids.

Method: assign_review_topics

Assigns reviewers to topics based on matched preferences and clears previous assignments.

Method: assign_topic_to_reviewer

Assigns a single topic to a reviewer by linking them to a team under that topic.

Method: reviewer_bidding_data

Retrieves bid details for a specific reviewer, including self-selected topics and bid priorities.

Method: fallback_algorithm

Assigns topics to reviewers using a fallback approach when the initial bidding fails.

Private Methods
  • sorted_topic_queue – Sorts topics based on the number of team members and returns them in descending order
  • assign_topics – Iterates through reviewers and assigns topics from the sorted queue, ensuring balanced distribution
  • find_available_topic – Selects an available topic for a reviewer while avoiding self-selected topics
  • reviewer_team_id – Retrieves the team ID associated with a given reviewer
  • find_reviewer_user_id – Fetches the user ID for a given reviewer
  • fetch_self_topic – Gets the topic that the user has self-selected for an assignment
  • fetch_team_id – Finds the team ID for a given user in an assignment
  • fetch_team_bids – Retrieves all bid records associated with a specific team
  • format_bid – Converts a bid record into a structured format with topic ID, priority, and a formatted timestamp

Service

The ReviewBiddingAlgorithmService is a service object responsible for handling communication between the application and the external web service that executes the review bidding algorithm. It prepares bidding data, sends it to the web service, processes the response, and provides a fallback mechanism in case of failure.

This service allows to encapsulate all logic required to interact with the external topic-matching web service. It separates the algorithm execution concerns from the controller, improving modularity, testability, and maintainability of the code.

Method: self.process_bidding

This is the primary entry point for the controller. It accepts the assignment_id and a list of reviewer_ids, then prepares the bidding data by calling ReviewBid.bidding_data(...). It passes the data to the web service by calling run_bidding_algorithm. If the web service is unavailable or fails, the method logs an error and falls back to an internal method: ReviewBid.fallback_algorithm. The result is returned as a hash mapping reviewer IDs to arrays of assigned topic IDs.

Method: self.run_bidding_algorithm

This method retrieves the review bidding web service URL from the app configuration (config/webservices.yml) and calls the internal send_bidding_request method to make the actual request.It logs the outgoing URL for debugging and delegates the response parsing.

Private Method: send_bidding_request

This private class method handles the HTTP POST request using RestClient. It sends the bidding data as JSON and expects a JSON response. If the request fails due to a connection error, server error, or timeout, the method logs the exception and returns false, allowing the caller to trigger a fallback.

]


Fallback Algorithm

Introduction

The Fallback Algorithm in Expertiza is designed to ensure automatic topic assignment when the primary Bidding Algorithm fails due to Web Service unavailability.

Normally, Expertiza allows reviewers to bid on topics, and the ReviewBiddingAlgorithmService processes these bids to assign topics. However, when this web service fails (due to API downtime, server issues, or unexpected errors), the system must switch to fallback and automatically assign topics to reviewers.

Solution

It is triggered when the web service is unavailable. It ensures fair topic assignment by:

  • Prioritizing topics with the largest teams.
  • Using a round-robin approach to distribute topics among reviewers.
  • Ensuring reviewers do not get their own team’s topic.
When is this Used?
  • The ReviewBiddingAlgorithmService fails due to an API issue.
  • The web service times out or returns an error.
  • Unexpected exceptions occur in the bidding process.

Instead of stopping the review process, the system automatically falls-back to an internal algorithm to distribute topics.

How Does the Fallback Algorithm Work?

The algorithm follows these **four main steps:

  • Fetch available topics – Retrieves all topics for the given assignment.
  • Sort topics by team size – Topics with more members are given priority.
  • Create a topic queue – Topics are arranged in descending order of team size.
  • Assign topics in a round-robin manner – Reviewers are assigned topics while avoiding their own team's topic.

This ensures fair and balanced distribution of topics. If ReviewBiddingAlgorithmService fails, the system automatically calls fallback_algorithm.

Implementation - Model Code (Fallback Algorithm)

This method ensures fair topic assignment when the bidding system fails.

Key Takeaway: This function ensures every reviewer receives a topic without requiring manual intervention.

RSpec Testing of Fallback Algorithm

To verify correctness, we use unit tests in review_bid_spec.rb

The Fallback Algorithm ensures that topics are assigned fairly even when the primary bidding service fails. By prioritizing larger teams, avoiding self-review, and using a round-robin strategy, the fallback mechanism keeps the review process smooth and automatic.


Test Plan

This section outlines the test plan for this project including testing scenarios/edge cases, the manual testing plan, and the automatic/RSpec testing plan.

Testing Scenarios

This section outlines the various bidding scenarios that could occur. These scenarios should be tested in either Manual Testing or Automatic Testing.

Basic Bidding Scenario

  • Reviewers bid on multiple projects at different times
    • Bidding algorithm should give priority to reviewers that bid first and should assign based on reviewers ranking

Edge Cases

  • All reviewers bid on the same project
    • If different time stamps: bidding algorithm should give priority to reviewers that bid first
    • If same time stamps: bidding algorithm should assign reviews randomly
  • None of the reviews bid on any projects
    • Bidding algorithm should assign reviews randomly
  • Reviewer bids on their own project
    • Bidding algorithm should not allow a reviewer to bid on their own project and should have validation to prevent assigning a reviewer their own project
  • None of the reviewers bid on a specific project
    • Bidding algorithm should still assign reviewers to this project


Manual Testing

Manual testing should be preformed on an Expertiza server and webserver platform

Manual Testing Plan for Expertiza Server

  • test that an instructor can allow review bidding for an assignment
  • test that server routes correctly when review bidding is allowed
    • test that task box links to bidding page before algorithm is run
    • test that assignment->others work links to bidding page before algorithm is run
    • test that task box links to review page with bidding information after algorithm is run
    • test that assignment->others work links to review page with bidding information after algorithm is run
  • test that the bidding UI is implemented as expected
  • test that a participant can bid on projects to review
    • test that a participant can move projects into the bidding column
    • test that a participant can reorder projects in the bidding column
    • test that a participant's bidding preferences save
    • test that a participant can't bid for their own topic
  • test that run algorithm button calls webserver with correct information and returns review assignments for each user

Directions for Manual Testing of Expertiza Server

1. Go to testing server
2. Log in as username: instructor6 password: password
3. Go to Manage -> Assignments -> E2085 Manual Testing -> edit (pencil symbol)

  1. In the General tab make sure Has teams? and Has topics? are checked
  2. In the Topics tab check Allow review to choose which topic to review? then click save
  3. In the Topics tab select Bidding then click save

4. Go to Assignments

  1. E2085 Manual Testing review task should be listed in the main assignments box
  1. This UI is based on instructor6 being a participant assigned to topic Topic4
  2. The UI blocks user from bidding on their own project
  3. Check that user can bid on topics and reorder bids
  4. Check that users topic preferences save on page refresh
  5. Impersonate other participants and repeat bidding UI tests ('student7601, student7602, student7603, student7604, student7605, student7606, student7607, student7608, student7609)

5. Go to Manage -> Assignments -> E2085 Manual Testing -> edit (pencil symbol)

  1. In the Topics tab click Run Review Algorithm button

6. Go to Assignments

  1. E2085 Manual Testing review task should be listed in the main assignments box
  1. E2085 Manual Testing review should now link to the reviewing page
  2. The number of required reviews should already be displayed
  3. A Request Another Review button should be present unless reviews listed is equal to the number of reviews allowed
  4. Begin a review and test save and submit

Manual Testing Demo Video for Expertiza Server

Video demonstration of manual testing: here

Automatic/PyTest Testing

Pytest tests are written for the webservice to test a well formed and a malformed request.

Automatic/RSpec Testing

RSpec tests will need to be refactored according to the changes that we make in the expertiza repository.


RSpec Testing Plan

These RSpec tests will need to:

  • test basic functionality of review bids controller
  • test basic functionality of review bid model

Directions for RSpec Testing

  1. on personal device using forked repo
  2. Run the following commands:
# review bids controller: 
bundle exec rspec spec/controllers/review_bids_controller_spec.rb

# review bid model: 
bundle exec rspec spec/models/review_bid_spec.rb

# review bidding algorithm service:
bundle exec rspec spec/services/review_bidding_algorithm_service_spec.rb

Test Coverage

Team

Mentor: Janice Uwujaren (juwujar)

Ashwin Muniswamy (akumarm)
Arturo Serdan (aaserdan)
Nicholas Winsen (nwinsen)

Resources

Previous Implementations

Fall 2019 here
Spring 2020 here
Fall 2020 here
Fall 2024 here

Relevant Links

Github Repository for Expertiza: https://github.com/ncsu-ngw/expertiza/tree/e2508-reimplement-bidding-algorithm-web-service
Github Repository for Web service: https://github.com/ncsu-ngw/BiddingService
Pull Request: https://github.com/expertiza/expertiza/pull/2931
Web Service Server: 152.7.178.10:8080/match_topics
Video Demo: https://app.screencast.com/vaiJnWpHf0r07

Retrieved from "https://wiki.expertiza.ncsu.edu/index.php?title=CSC/ECE_517_Spring_2025_-_E2508._Reimplement_bidding-algorithm_web_service&oldid=162743"