CSC/ECE 517 Spring 2016/M1704 Servo Dependency Tool

The Servo Dependency Tool is a set of python scripts that automatically upgrades Cargo dependencies for the Servo web browser engine.

Background

Servo is a prototype web browser engine created by Mozilla Research and written in the Rust programming language. It is currently developed on 64-bit OS X, 64-bit Linux, and Android platforms. Servo depends on numerous other Rust libraries that are published on the package manager crates.io. There are no notifications for when packages are updated so it's up to developers to keep track of when they need to upgrade the dependencies within their projects.

The goal of the Servo Dependency Tool is to automatically upgrade Servo's dependencies whenever new versions are released.

Requirements[1]

• Use Python, Node.js, or Ruby to develop the tool
• Tool must be easily ran on hosted services (Heroku, Amazon AWS, etc)
• Parse a Cargo.lock file to determine the list of dependencies and their current versions
• Compare the list of dependencies and their current versions to a local clone of the crates.io index to find out the latest versions available
• Edit all Cargo.toml files within Servo with the latest versions
• Update the packages by running the cargo-update command using Servo's Mach tool
• Interact with GitHub to create a branch for the updates and then open a pull request on the main Servo repo

Tool Workflow Overview

1. Perform a pull to ensure the servo fork is up-to-date
2. Create a new branch on the servo fork to contain the changes performed by the tool
3. Parse the Cargo.lock file to determine all the packages within Servo and their current versions
4. Make a local clone of the crates.io index and compare the current versions of each package to the most recent version in the crates.io index
5. Iterate through all the Cargo.toml files in the Servo project and update all listed versions to the most recent version found in the previous step
6. For each package with an update, run the cargo-update command using the Mach tool
7. Push all updates to the newly created upstream branch
8. Submit a pull request containing these updates to the main Servo repository

Project Structure

servo_dependency_tool.py

servo_dependency_tool.py is the main driver file of our tool. This file imports all of the other files and controls the overall flow of the tool from start to finish.

cargo_lock_parser.py

This module handles parsing Cargo.lock files to get metadata on project dependencies and store in Python objects. The purpose of Cargo.lock files are to store data on dependencies for Rust packages. cargo_lock_parse completes this by creating several class objects. LockFile represents the project's Cargo.lock file. LockRoot represents the root from the root tag which is present in all Cargo.lock files. LockPackage stores package information for version, an upgrade is available, and dependencies. A package can have 0 or more dependencies that come from a specific source and are stored in a LockDependency object.

crates_io_checker.py

The purpose of this module is to check each package in the crates.io repository to see if an upgrade is available. It clones the latest version of the crates.io Github repository, checks if the package is in the crates index, and determines if the package version is the latest version.

cargo_toml_updater

This module updates all Cargo.toml files within the project to display the latest version. In order for the upgrades to take place, the .toml files for dependencies must be updated in the repository. TOML is a markup language that is minimal, easy to read, maps key and value pairs similar to a hash table, and has been implemented in various programming languages.

run_cargo_update.py

Text

repo_management.py

The repo_management module is focused on handling the Github operations for project repositories. The code depends on external Python Github APIs gitpython and github3. It handles pulling the master branch of a project repo, creating new a new branch, pushing to the recently created branch, and creating a pull request on the repository.

Running the Tool

The Servo Dependency Tool has been designed to be easy to setup and use. Detailed information can be found in the README on GitHub.

Prerequisites

The tool runs using Python3 and requires two python libraries in order to run: GitPython and github3.py. In addition, it is expected that the machine running the tool has a local copy of the forked Servo repo.

Installation and Execution

Installing the tool requires cloning the Servo Dependency Tool into the root folder of the local copy of Servo:

   git clone https://github.com/chbrown13/servo-dependency-tool

Call servo_dependency_tool.py to run the tool:

   python3 servo_dependency_tool.py

Testing

Tests for the Servo Dependency Tool were implemented using the Python 3 unit testing framework. These tests were created to ensure certain functionality of the tool works appropriately and compares actual values to expected output. The unit tests can be found in the test folder of the project's repository. To run the unit tests, use the following command:

   python3 -m unittest <module-name>

Project Links

• Servo Dependency Tool (Our GitHub): https://github.com/chbrown13/servo-dependency-tool
• Project Request Link: https://github.com/servo/servo/wiki/Cargo-upgrade-service-project
• GitHub Issue Tracker: https://github.com/servo/servo/issues/15600