UROS — An AI That Actually Learns You

What if your AI didn't just remember what you said, but actually learned from you?

Inspiration

Imagine having an AI assistant that gets to know you the same way another person would. The more you talk, the better it understands how you think, how you communicate, and what matters to you.

Today, AI personalization largely depends on system prompts, saved instructions, or retrieving information from previous conversations. The underlying intelligence often remains unchanged.

We wanted to change that.

What if your AI could learn your communication style simply through conversation? What if it could recognize when you wanted a shorter answer, a different explanation, or more detail without requiring you to constantly rate its responses?

And what if the things it learned could actually become part of its neural-network weights?

That question inspired UROS, an AI architecture designed to learn continuously from individual users through preference optimization and parametric memory.

The name itself represents our vision. UROS is a Finnish word with an older, poetic meaning of "hero." It is also our shorthand for "Your OS" — Your Operating System.

We believe everyone deserves an AI that works for them, learns from them, and gives them greater control over their own intelligence.

We set out to explore something more ambitious than another chatbot with a memory database:

An AI that becomes increasingly unique to its user through actual machine learning.

What We Built

UROS combines three different learning systems, each designed to operate on a different timescale. Together, they form the foundation for a personalized intelligence that learns from conversations, updates its neural-network parameters, and preserves what it has learned.

1. Personalized Reward Model (PRM) — Learning How You Think

UROS can learn communication preferences directly from natural conversations without requiring constant thumbs-up ratings or A/B comparisons.

If you say "make that shorter," "give me an example," or "that's not what I meant," UROS identifies potential preference signals and uses them to improve its understanding of how you like to communicate.

Our Personalized Reward Model (PRM) evaluates possible responses using interpretable characteristics such as:

Response length, directness, and structure
Specificity, examples, and level of detail
Context-dependent communication preferences
Feedback patterns across separate conversations

The PRM updates its own reward-model parameters immediately when meaningful feedback arrives. This allows UROS to evaluate and rank responses according to the individual's learned preferences.

2. DPO + LoRA — Teaching the Language Model

Rather than relying exclusively on instructions telling the AI how to respond, UROS can actually change the neural-network parameters responsible for generating answers.

We implemented Direct Preference Optimization (DPO) and Low-Rank Adaptation (LoRA) to train the language model using accumulated preference evidence.

Our background training system supports incremental learning, periodic eligibility checks, and continued training from previously accepted adapters.

Instead of blindly applying every update, UROS evaluates candidate adapters against held-out questions and general capability benchmarks.

Updates that fail the evaluation can be rejected, while successful versions can be saved and activated.

This gives UROS a mechanism for turning conversational feedback into longer-lasting behavioral learning.

3. MEMIT — Giving an AI Parametric Memory

This was one of the most exciting parts of our project.

What if an AI could remember personal information without searching through old messages?

We explored two approaches to learning facts directly in model parameters: intensive LoRA supervised fine-tuning and MEMIT (Mass-Editing Memory in a Transformer).

We implemented and integrated a MEMIT-style factual memory editor for SmolLM2-1.7B-Instruct.

Rather than training the entire model on repeated examples, MEMIT computes targeted changes to selected neural-network weights, creating factual associations inside the model.

Our implementation supports:

Multi-layer factual editing
Reversible neural-network weight updates
Per-user memory versions
Checkpoint integrity verification
Compatibility with existing LoRA personalization
Memory-preservation and rollback mechanisms

We also built a nightly memory workflow that processes eligible conversation records, prepares factual candidates for review, and can train approved memories.

This provides the foundation for long-term memory consolidation without depending on retrieval at inference.

Together, PRM, DPO, LoRA, and MEMIT give UROS multiple ways to learn from one person, rather than treating every conversation as an isolated interaction.

Our Unique Value Proposition — What Makes UROS Different?

Most AI personalization changes the information a model receives. UROS is designed to change what the model has actually learned.

Our innovation isn't any single algorithm. It's the architecture we've built around multiple learning methods and the way they can complement each other.

1. Personalization Without Constant Explicit Feedback

Traditional preference-training systems often rely on users manually choosing between responses, providing ratings, or generating labeled datasets.

We built a system that can identify preference evidence from natural conversation.

The conversation itself becomes a potential learning signal.

Our PRM can recognize meaningful corrections and recurring preferences, update its learned reward parameters, and use that information to improve response selection without constantly interrupting the user.

2. Turning Conversations Into Persistent Neural-Network Learning

UROS connects short-term preference recognition with longer-term model training.

Rather than limiting personalization to a system prompt, we built a pipeline that can transform accumulated preference evidence into actual LoRA weight updates.

This creates a pathway from ordinary human communication to changes in the language model's behavior.

The user doesn't just customize the AI. Their interactions can help train it.

3. Factual Memory Without Retrieving Previous Conversations

One of our most important experimental achievements was demonstrating that personal facts could be encoded into model parameters and recalled from previously unseen questions without retrieving the original conversation.

We demonstrated this through intensive LoRA fine-tuning and expanded the architecture with a MEMIT-style factual editing system.

These are complementary approaches to the same larger goal: enabling AI memory that is learned rather than merely looked up.

4. Combining MEMIT With Existing LoRA Personalization

MEMIT and LoRA solve different problems, and combining them inside a personalized assistant presents significant engineering challenges.

We implemented a system that allows targeted factual edits to coexist with preference-trained LoRA adapters.

Our architecture uses reversible memory-edit factors and per-user activation, allowing different users to have distinct learned information while sharing the same underlying base language model.

This avoids requiring a completely separate 1.7-billion-parameter model for every user.

We aren't just personalizing an AI's responses. We're exploring how to personalize its effective knowledge and behavior together.

5. Learning That Must Earn Its Place

A model changing its weights doesn't automatically mean it has improved.

We made this a central design principle.

UROS includes held-out evaluation gates, model-version tracking, checksum verification, and rollback mechanisms. Candidate LoRA updates can be rejected if they fail to demonstrate sufficient improvement or cause unacceptable regressions.

The result is an architecture where learning can be inspected and measured rather than blindly trusted.

6. Multiple Learning Timescales in One Personal AI

Our system brings together different learning mechanisms that operate at different speeds:

Immediate: PRM learns from meaningful conversational feedback.
Periodic: DPO and LoRA consolidate reliable preferences into model parameters.
Nightly: MEMIT workflows prepare and edit approved factual memories.

We have implemented the underlying learning components and nightly review-and-training workflow. Fully automated scheduling and reliable fact approval remain areas for further development.

Our central innovation is the integration of conversational preference learning, parameter-efficient model training, and targeted factual memory editing into one user-controlled AI architecture.

How We Built It

Our technology stack includes:

AI/ML: Python, PyTorch, Hugging Face Transformers, PRM, DPO, LoRA, SFT, and MEMIT-style weight editing
Language model: SmolLM2-1.7B-Instruct
Backend: FastAPI
Frontend: React and TypeScript
Authentication: Auth0
Hardware: NVIDIA L4 GPU on a university HPC cluster
Deployment: Server-hosted inference and training through an SSH tunnel

Training the Model With LoRA

One of our biggest engineering decisions was using Low-Rank Adaptation to train a language model without modifying every original parameter.

LoRA introduces smaller trainable matrices into the existing network:

$$ y = Wx + \frac{\alpha}{r}B(Ax) $$

Where:

(W) is the original frozen weight matrix.
(A) and (B) are trainable low-rank matrices.
(r) represents the adapter rank.
(\alpha) controls the strength of the adaptation.

By training these smaller matrices, we can change how the model responds while preserving its original parameters.

We implemented both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO).

SFT helps the model generate preferred answers, while DPO trains on comparisons between chosen and rejected responses.

The DPO objective can be expressed as:

$$ \mathcal{L}_{DPO} = -\log \sigma\left( \beta \left[

\log \frac{\pi_\theta(y_w \mid x)}{\pi_\theta(y_l \mid x)}

\log \frac{\pi_{\mathrm{ref}}(y_w \mid x)}{\pi_{\mathrm{ref}}(y_l \mid x)} \right] \right) $$

Our PRM uses a Bradley–Terry-style preference model with interpretable features, contextual weighting, and uncertainty estimates.

Building MEMIT Memory Editing

For our memory system, we implemented a Llama-architecture adaptation of the MEMIT algorithm.

The idea is to calculate targeted updates to a model's effective weight matrices:

$$ W_{\text{edited}} = W + \Delta W $$

Our implementation optimizes target factual representations and distributes calculated changes across multiple transformer layers.

To make this practical for the 1.7B model, we developed a low-rank covariance approximation that avoids constructing extremely large covariance matrices.

We also engineered reversible memory-edit overlays, allowing different users to have different learned factual associations while sharing the same underlying language model.

This was particularly important because UROS already uses LoRA for personalization.

We needed a way for the memory-editing system and preference-learning system to coexist without overwriting one another.

Making Learning Safe and Measurable

We developed infrastructure to manage a continuously learning AI.

UROS includes:

Versioned LoRA adapters
Checkpoint and checksum verification
Held-out evaluation gates
Model rollback mechanisms
User-specific memory isolation
Private conversations that bypass personalization and learning
Controls to inspect, pause, and delete learned information

Rather than treating every decrease in training loss as an improvement, our system can reject updates that don't demonstrate meaningful progress.

Challenges We Faced

Getting the Model to Actually Learn

Our first experiments used a 135-million-parameter model.

While it demonstrated some interesting learning behavior, it struggled with factual reliability and following instructions.

We moved to SmolLM2-1.7B-Instruct and optimized our training infrastructure to run on an NVIDIA GPU.

That gave us a significantly more capable foundation while introducing new challenges involving training time, GPU memory, and efficient parameter updates.

Turning Feedback Into Better Responses

One surprising discovery was that minimizing training loss doesn't always improve the model's behavior.

In one DPO experiment, our loss decreased to approximately 0.05, but the model still struggled to consistently produce shorter answers.

We experimented with supervised fine-tuning, training objectives, and response formatting to improve how the model learned preferred communication styles.

We learned that lower loss doesn't necessarily mean better intelligence.

Teaching an AI to Remember Without Retrieval

Our earliest memory experiments were fascinating because the model sometimes recognized a correct answer when presented with multiple choices but couldn't generate that answer independently.

We adjusted the LoRA configuration, training intensity, question paraphrasing, and prompt consistency to improve free-form recall.

We then took that research further by implementing MEMIT-style editing, moving from repeated gradient-based learning toward targeted factual weight updates.

Combining Three Different Learning Mechanisms

PRM, DPO, and MEMIT work differently.

The PRM learns preference rankings, DPO adjusts language-generation behavior, and MEMIT edits factual associations.

Making those systems compatible required careful handling of weights, checkpoints, user isolation, concurrency, and rollback.

We also strengthened privacy protections, ensuring Private mode doesn't record conversations or apply stored personalization.

One of our biggest engineering lessons was that building an AI capable of learning is only half the challenge.

Making sure it learns the right things, retains them, and doesn't lose other capabilities is just as important.

Accomplishments That We're Proud Of

We Trained a Real 1.7-Billion-Parameter Language Model

Our server-side LoRA experiment successfully trained:

18.1 million trainable parameters, approximately 1.06% of the model
336 updated adapter tensors
Approximately 72 MB of saved adapter weights
No changes to the original frozen base-model weights

This demonstrated actual neural-network training, not simply prompt engineering or an external memory system.

We Demonstrated Measurable Behavioral Learning

In our held-out conciseness experiment:

Model	Concise Responses
UROS fine-tuned adapter	15/16 (93.75%)
Base SmolLM2 model	0/16 (0%)
Base model with a conciseness prompt	12/16 (75%)

Our trained adapter also preserved correct responses on 12/12 factual questions in that evaluation.

The evaluation exposed one instruction-following regression, reinforcing why we built capability-preservation tests alongside preference benchmarks.

We Demonstrated Factual Memory Inside Neural-Network Parameters

In a separate experiment with SmolLM2-135M, we trained an adapter to remember fictional personal facts.

On held-out paraphrased questions, we achieved:

8/10 correct free-form answers from the memory-trained model
0/10 correct free-form answers from the original model
10/10 correct forced-choice answers from the trained model

Crucially, the model received no retrieved memories or previous conversation history during inference.

We also verified that the learned adapter could be saved, reloaded in a separate Python process, and still recall the trained information.

Although this was an initial small-scale benchmark, it demonstrated the possibility of encoding personal memories directly into trainable neural-network parameters.

We Built and Integrated a MEMIT Memory Editor

We implemented a MEMIT-style system for targeted factual weight editing on the 1.7B architecture and integrated it into UROS's main codebase.

Our implementation includes:

Multi-layer factual editing
Memory-efficient weight-update calculations
Reversible per-user memory edits
Persistent memory checkpoints and integrity verification
Compatibility with existing LoRA and DPO components
A nightly workflow for reviewing conversations and training approved facts

The implementation passed its initial local tests, and large-model CUDA evaluations are the next step in validating its memory-editing accuracy.

We Built a Learning System With Safeguards

UROS includes an evaluation gate capable of rejecting underperforming adapters, versioned learning state, rollback, user-specific memory isolation, and privacy controls.

Our larger testing infrastructure includes hundreds of automated tests.

We're proud not only that we demonstrated learning, but also that we developed mechanisms to measure, inspect, and control what the system learns.

What We Learned

Our biggest takeaway is that AI personalization doesn't have to exist only in prompts, conversation histories, or retrieval databases. It can happen inside the neural network itself.

But learning is more complicated than simply modifying weights.

We learned that different types of learning require different techniques.

A Personalized Reward Model can quickly recognize communication preferences. DPO and LoRA can consolidate preferences into a language model's behavior. MEMIT offers a different approach to targeted factual memory.

We also learned that strong evaluation is essential.

Prompting can already be a surprisingly competitive personalization method, and early broader experiments using real PRISM participant preferences showed that improvements don't automatically generalize across users.

In one experiment using 92 held-out preference comparisons, we observed modest differences between the base and personalized models, but no statistically significant improvement over the base model.

That led us to focus on held-out tests, capability preservation, and rejecting changes that don't earn their place.

Perhaps our most important realization was that:

The future of personalized AI may not be one learning algorithm, but multiple complementary learning systems working together.

UROS is our attempt to bring those systems into one architecture.

What's Next for UROS?

Our next priority isn't simply adding more features.

It's finding out how far we can push personalized learning while making the system dramatically more capable, reliable, and safe.

1. Scale to More Powerful AI Models

We've built our foundation around SmolLM2-1.7B, but we want to expand our experiments to larger and more capable open-weight language models.

We plan to investigate how model size, architecture, and training capacity affect personalized learning, factual memory, response quality, and resistance to forgetting.

Our goal is to discover whether these methods become more effective as models become more intelligent, and whether we can achieve significantly stronger results across different architectures.

2. Aggressively Pursue AI Safety

A system that continuously modifies its own effective weights introduces important safety questions.

We want to rigorously test for:

Catastrophic forgetting: Does learning new information erase older knowledge?
Hallucinations: Can the system recognize when it doesn't know something?
Conflicting memories: What happens when newly learned information contradicts previous information?
Privacy leakage: Can personalized memories accidentally appear in another user's responses?
Adversarial manipulation: Can malicious conversations influence what the model learns?
Model degradation: Can repeated personalization damage reasoning, factual accuracy, or instruction following?

We also want to investigate prompt injection, poisoned preference signals, and whether repeated personalization could cause a model to become less truthful, less reliable, or more easily manipulated.

We want UROS to become better at understanding its user without compromising the safeguards that make an AI trustworthy.

Our goal is not just continuous learning.

It's safe, measurable continuous learning.

3. Push for Exceptional Results

We want to move beyond small benchmarks into larger, statistically meaningful experiments.

That means testing:

Hundreds or thousands of factual associations
Larger and more diverse participant datasets
More powerful language-model architectures
Multiple consecutive rounds of memory editing
Long-term preference retention
Cross-model generalization
Safety and capability preservation after repeated updates

We'll compare PRM, DPO, SFT, MEMIT, and hybrid approaches, measuring what each method contributes and how well they work together.

We also want to investigate whether a model can accumulate knowledge over long periods while preserving the reasoning ability, factual accuracy, and instruction-following behavior it started with.

We're not just trying to show that AI can learn. We're trying to discover how well it can learn, how much it can retain, and how safely it can improve.

4. Make Personal AI More Efficient and Accessible

Longer term, we want to reduce training and inference costs, improve the efficiency of personalized adapters, and investigate deployments on consumer hardware.

Our current research runs on server GPUs, but we see opportunities for smaller and more efficient models to bring this type of personalization closer to the user.

Our Vision

We started this project with a simple question:

What if your AI could actually learn you?

Now we're exploring something even bigger.

What happens when a language model learns from thousands of conversations, develops a personalized understanding of one individual, and continuously updates its own parameters over months or years?

Imagine an assistant that understands how you communicate, remembers the things that matter to you, adapts when your preferences change, and continually improves while preserving your control over what it learns.

Not a generic AI with your name attached.

Not a chatbot that simply searches through your history.

An intelligence that becomes uniquely yours.

That's what we're building with UROS.

View UROS on GitHub