Goal
In this challenge, you will create an interactive human-in-the-loop training system using a simple machine learning model.
You are given parts of a web application and must implement the backend for this system
The frontend (shown above)
Allows users to interactively add data points to a graph
Sends data points to the backend
Receives model parameters from the backend and plots a density over the points
The backend must
Accept these data points and train the model
Return the parameters of the model to the front-end
Detailed requirements
Read through the provided code
Implement a backend API service that hosts the model, trains it, and returns model parameters
When `p` is pressed, you need to retrieve the parameters of the model and pass them to the frontend for plotting
Implement a logging solution for the model so that you can keep track of:
training loss
model versions
data used to train each model
Implement a solution to visualize each training result
Submit a README.txt containing details about the methods you implemented, problems you ran into, and how to improve your submission if you had more time
Provided code
Web application:
solution_skeleton.py: web application. Root endpoint takes the user to the app shown above. You’ll need to implement any other endpoints in this file
© Quilt Labs, Inc. | 2023
index.html: static HTML page with layouts for visual elements of the app
static/script.js: Contains plotting code for the web app
Model training code:
gmm.py: contains Pytorch model class, function to train the model, function to extract parameters from the model and return them as lists of floats
Judgment criteria
Does your web app work? For your submission to be considered complete, it must run on our Ubuntu 20.04 VM. If you choose to include additional requirements, please be sure to mention them in your README.txt
How clean is your code? Do you use comments? Type annotations?
Do you discuss testing in README.txt? How would you think to test an application like this?
How did you implement logging and model versioning? How do you store metadata associated with a trained model?
How easy to understand is your visualization of training results?
How memory-efficient is your code? What is the runtime of a training run?
FAQs
Which files can I edit? Anything in the folder can be edited
Can I use other packages? Yes, just list them in your README.txt so we can install them before judging
Do I need to worry about cleaning up visualizations? Not really, as long as they’re easy to understand
How should I submit the code? Please zip your submission and upload to Google Drive, then share a public link with us via email
© Quilt Labs, Inc. | 2023
