Trustworthy Model Registry
Group project building a model registry with AWS deployment. Led AWS infrastructure setup (ECS/Fargate) and implemented three evaluation metrics: Reproducibility, Reviewedness, and Treescore.
Architecture
Model registry system with AWS ECS/Fargate deployment. Containerized application with CI/CD pipeline. Three evaluation metrics: Reproducibility, Reviewedness, and Treescore for model trustworthiness assessment.
Model Registry → AWS ECS/Fargate → Docker → CI/CD → Evaluation Metrics → Trust ScoreKey Features
AWS Infrastructure
Containerized application infrastructure using AWS ECS and Fargate, including task definitions, security groups, and networking configuration.
Metric Implementation
Implemented three evaluation metrics: Reproducibility (example code validation), Reviewedness (PR review analysis), and Treescore (parent model lineage scoring).
Team Collaboration
Worked in simulated software engineering environment with weekly milestones, client requirements, and team-based development practices.
Model Registry System
Built model registry with CRUD operations, search functionality, lineage tracking, and model evaluation capabilities.
Technical Details
AWS ECS/Fargate Deployment
Containerized deployment with Docker, ECR repositories, task definitions, and automated CI/CD pipelines via GitHub Actions.
Evaluation Metrics
Reproducibility: Analyzes model repositories for example code availability and validity. Reviewedness: Calculates fraction of code added via reviewed pull requests. Treescore: Computes average net score of parent models in lineage graph.
Impact & Results
Demonstrated cloud infrastructure expertise and metric design skills in team-based software engineering project
Key Achievements
Designed and deployed AWS ECS/Fargate infrastructure for containerized application hosting
Implemented Reproducibility metric analyzing example code availability and validity
Implemented Reviewedness metric calculating fraction of code added via reviewed pull requests
Implemented Treescore metric computing average net score of parent models in lineage graph
Simulated real software engineering workflow with weekly milestones and client requirements
Technical Innovation
- • Metric design and implementation for model evaluation
- • Container orchestration with AWS ECS/Fargate
- • CI/CD pipeline automation with GitHub Actions
- • Team-based software engineering practices
- • Model registry architecture and data management