What it does
MLM Classifier turns a small labeled text dataset into a tested, downloadable private classifier. A user starts with seed labels, a fixed evaluation set, and unlabeled documents. The application trains a real local baseline, selects useful uncertain or underrepresented examples, asks the user to confirm every label, retrains a candidate, and promotes it only when holdout macro F1 improves.
The included annual-report profile identifies balance sheets and income statements, but the workflow is domain-independent. Profiles for banking credit, insurance claims, contracts, and support tickets show how the same tool applies wherever meaningful labeled datasets are scarce.
Human-controlled continuous learning
The system supports uncertainty, rare-class, diversity, hard-negative, confusion-pair, and profile-term acquisition strategies. GPT-5.6 can choose the strategy from a bounded schema or selectively teach on difficult candidates. Deterministic application code validates its output. GPT cannot execute code, fetch a source, add a label, retrain, or deploy a model by itself.
The OTHER class has an explicit balance policy: by default MLM requests at least one confirmed hard negative for every two positive examples. Search results are suggestions only; a user must select exact public sources and confirm before anything is fetched. Endpoint feedback also waits in a review queue until approved.
Real model lifecycle
Each cycle fits a local TF-IDF and logistic-regression student on seed plus confirmed replay labels. It evaluates the candidate against a fixed holdout set, registers an immutable model version, and changes production only when macro F1 improves. A weaker model remains auditable and is not deployed.
Production inference runs locally without an LLM. The user can download the complete dataset and standalone model bundles, including manifests and SHA-256 checksums. A local installation can also expose FastAPI prediction and feedback endpoints.
How it was built
Codex was used to extend the original machine-learning architecture into this complete product: the guided Gradio interface, project lifecycle, live labeling, class-balance controls, confirmation gates, evaluation and promotion flow, endpoint feedback review, portable exports, automated tests, and deployment packaging.
GPT-5.6 is an optional runtime component for acquisition planning, selective teaching, and public-source discovery. Core labeling, model training, evaluation, inference, downloads, and endpoint serving work without an OpenAI API key.
Testing
The public repository includes one-command Windows and macOS/Linux launchers, Docker configuration, judge instructions, and deterministic checks. The release was verified through the browser from baseline fitting through live labeling, imbalance correction, candidate rejection, local inference, and generation of both download bundles. The automated suite covers model serialization, promotion and rejection, confirmation-gated acquisition, export portability, endpoint inference, and feedback review.
Built With
- codex
- fastapi
- gpt-5.6
- gradio
- hugging-face
- openai-api
- python
- scikit-learn