Sirius Logo
Sirius Architecture
Home Screen Widget
Library
iOS Calendar

Inspiration

As first-generation and immigrant students, we juggle busy lives jobs, family, commute times, and heavy course loads. Traditional study apps require us to sit down for long, isolated hours that we rarely have. We noticed that while social media apps capture our attention effortlessly through micro-interactions throughout the day, study tools remain passive and bulky. We built Sirius to transform dead mobile downtime into continuous, ambient learning. Our goal was to make studying more accessible, interactive, and consistent while giving students a companion that helps them actually understand their material rather than just memorize answers.

Studying shouldn't feel like a chore, and learning shouldn't require students to sit in front of a screen for hours.

We were inspired by a simple question: What if studying felt more like having a conversation with a friend who actually understands your coursework?

Students already have the resources they need to learn: textbooks, lecture notes, and PDFs. But turning hundreds of pages into something understandable, engaging, and memorable takes time. Traditional study tools often require students to manually create flashcards, summarize chapters, or type questions into a chatbot.

We wanted to change that.

That's why we built Sirius, an open-source, voice-first AI study companion for iPhone and iPad that transforms students' own textbooks into personalized podcast-style lessons, interactive quizzes, and scheduled study sessions.

Instead of typing into a chat box, students simply wake the Sirius cube and start talking. Sirius listens, understands what they're studying, and helps them learn naturally.

Our goal was to make studying more accessible, interactive, and consistent while giving students a companion that helps them actually understand their material rather than just memorize answers.

What it does

Sirius turns your textbooks into conversations, podcasts, quizzes, and actionable study plans.

At its core, Sirius is a voice-first learning experience powered by Gemini Live. Students interact with an animated Sirius cube, speaking naturally instead of navigating through traditional chatbot interfaces.

Sirius supports an entire learning workflow:

🎙️ Natural voice conversations

Using Gemini Live, students can speak directly to Sirius, interrupt it mid-conversation, ask follow-up questions, and navigate their learning experience completely through voice.

📚 Learn from your own textbooks

Students import PDF textbooks and organize them into classes. Sirius identifies chapters, subsections, and page ranges, allowing it to generate learning content grounded in the student's actual course materials instead of relying on generic AI knowledge.

🎧 Turn chapters into podcasts

Sirius transforms textbook sections into engaging, podcast-style lessons narrated using ElevenLabs. With background audio playback, students can learn while walking, commuting, or doing everyday activities.

🧠 Test your knowledge with adaptive quizzes

Sirius generates 15-question quizzes that progress from easier concepts to more challenging ones. Questions include multiple choice, yes/no, and checkbox formats.

If students get something wrong, Sirius can provide contextual hints and additional remediation questions to reinforce concepts without simply revealing the answer.

📅 Build a realistic study schedule

Sirius can check calendar availability and suggest study sessions based on the student's schedule. After receiving explicit approval, it creates calendar events and local notifications that link directly to the corresponding lesson or quiz.

📱 Pick up exactly where you left off

Learning progress is saved locally using Core Data. Sirius remembers classes, activities, bookmarks, and study checkpoints.

A Home Screen widget displays the next scheduled activity, while deep links allow students to jump straight into a specific lesson or quiz from their calendar, widget, or notification.

The result is more than an AI tutor. Sirius connects learning, practice, and time management into one continuous experience.

How we built it

We built Sirius as a native iOS application using Swift, SwiftUI, SceneKit, Google Gemini, ElevenLabs, and Apple's native frameworks.

The architecture follows one central principle: Gemini decides what should happen. Deterministic tools decide how it safely happens. Student-provided materials decide what Sirius is allowed to teach.

Rather than allowing a generative AI model to handle everything, we separated conversational intelligence, long-form content generation, and native application execution into different systems.

Google Gemini Live API — Real-time conversational intelligence

Gemini Live powers Sirius's voice-first interface.

We implemented bidirectional audio communication over WebSockets, using AVFoundation to handle microphone capture, voice processing, PCM audio conversion, and playback.

Students can speak naturally, interrupt responses, and interact with their study materials conversationally.

Gemini Live also acts as the orchestration layer, invoking narrowly defined tools to discover classes, select textbooks, extract PDF chapters, resume activities, and initiate study planning.

The model doesn't receive the entire local database. Instead, it retrieves only the information needed for the current task.

Gemini REST API — Structured, source-grounded generation

We use Gemini REST's generateContent API separately from Gemini Live to create longer educational content. Once a student selects a chapter, Sirius extracts the exact source text from the PDF and passes it to Gemini along with a defined JSON schema.

Gemini generates structured lessons and 15-question quiz sets. Before saving anything, Sirius validates the generated output against requirements such as lesson structure, question counts, answer options, source excerpts, and metadata.

Separating real-time conversation from larger generation tasks keeps the voice interaction focused while giving long-form content generation its own reliable pipeline.

PDFKit — Turning textbooks into structured learning materials

PDFKit allows Sirius to inspect document bookmarks, identify chapter hierarchies, and extract exact page ranges. We built deterministic PDF tools to resolve a student's requested chapter or subsection and verify that the required source text has been extracted.

That content becomes the grounding material for lessons, quizzes, and explanations.

ElevenLabs — Generating podcast-style narration

Once Gemini generates and validates a lesson, we save the readable lesson before starting audio generation. We then send individual lesson segments to ElevenLabs for narration and store the resulting MP3 files locally. This separation is intentional: if audio generation fails, the underlying lesson remains available instead of being lost.

For playback, we integrated AVFoundation and MediaPlayer to support background audio, lock-screen controls, and Now Playing information.

SceneKit and SwiftUI — Creating Sirius's interface

We used SwiftUI to build the application experience and SceneKit to create the animated 3D Sirius cube. Rather than centering the app around a text conversation, the cube acts as the visual identity of the voice assistant and helps make interactions feel more personal. Combine helps coordinate observable application state across the interface and learning systems.

Core Data — Local-first learning persistence

Core Data stores learner profiles, classes, textbooks, lessons, quizzes, progress checkpoints, and generation operations. Imported PDF files and generated MP3s are stored in the application's local file system.

Every generated activity has stable identifiers that make it possible to resume progress and link directly to the correct material. We don't use MongoDB or an external cloud database for student records.

EventKit, notifications, and WidgetKit, LiveActivity Kit— Making learning consistent

We integrated Apple's calendar functionality to check availability and create approved study sessions. EventKit lets Sirius check calendar availability and schedule approved study sessions.

UserNotifications handles activity-specific reminders, while WidgetKit and ActivityKit extend the learning experience outside the main application through widgets and Live Activities.

We also implemented custom sirius:// deep links so calendar events, notifications, and widgets all navigate back to the correct lesson or quiz.

Privacy and security

Student materials, generated audio, profiles, and progress are stored locally. API credentials are secured in the iOS Keychain, and calendar events are created only with explicit student approval.

The app communicates directly with Gemini and ElevenLabs for AI-powered functionality, while retaining local control over the learning experience. We also implemented idempotent operation IDs, tool-call deduplication, bounded retries, and explicit error handling so that repeated model requests don't accidentally create duplicate activities or events.

Together, these systems allow Gemini to provide the intelligence of the experience while native iOS tools maintain predictable application behavior.

Challenges we ran into

1. Making real-time voice interactions feel natural

Integrating Gemini Live into a native iOS experience involved more than sending audio to a model.

We needed to coordinate microphone input, streaming responses, interruptions, and state changes while keeping the interaction smooth and responsive.

A voice-first interface also meant we couldn't rely on traditional buttons and menus to resolve every ambiguous request.

2. Keeping generated content grounded in textbooks

One of the most important challenges was making sure Sirius generated learning material from the student's actual PDFs.

Textbooks contain complex structures, including nested bookmarks, chapters, and subsections. Extracting the correct pages and preserving their relationships was essential.

We addressed this by building deterministic PDF extraction tools that verify source sections before generation.

3. Connecting AI decisions to reliable app actions

Language models are great at understanding intent, but operations like updating progress, retrieving saved activities, and creating calendar events need predictable behavior.

We solved this by separating conversational reasoning from local tool execution, using structured interactions and stable identifiers to prevent the AI from having to guess application state.

4. Building a complete learning loop

Generating a lesson is only one part of studying.

We needed quizzes, progress tracking, hints, remediation, scheduling, reminders, and navigation to work together without losing context.

Connecting these systems required careful coordination between Core Data, calendar events, notifications, widgets, and deep links.

5. Balancing AI capabilities with user privacy

We wanted Sirius to feel intelligent without unnecessarily exposing student information.

This meant keeping learning data local, securing credentials in Keychain, limiting the context shared with the AI, and requiring confirmation before actions that change a student's calendar.

Accomplishments that we're proud of

Our biggest accomplishment is building an end-to-end voice-first study experience, rather than another chatbot wrapped around an AI API.

We're especially proud that:

Sirius is truly voice-first. Students can navigate their learning experience through natural conversation with Gemini Live.
Learning is grounded in real materials. Sirius extracts specific PDF chapters and page ranges rather than generating generic lessons disconnected from coursework.
Textbooks become reusable audio lessons. ElevenLabs narration turns dense reading material into podcast-style content with background playback.
Quizzes encourage understanding. Progressive difficulty, contextual hints, and remediation rounds help students practice concepts rather than just collect correct answers.
Studying extends beyond the app. Calendar integration, local notifications, Home Screen widgets, and deep links create a connected learning experience.
Student progress stays persistent. Classes, lessons, quizzes, and checkpoints are stored locally, allowing students to resume where they left off.
The project is open-source. Other developers can inspect, improve, and contribute to Sirius.

We're particularly proud of how the different technologies work together. Gemini provides the conversation, ElevenLabs gives lessons a voice, and native iOS frameworks make the experience feel like a cohesive study companion.

What we learned

Building Sirius taught us that creating a useful AI product isn't just about choosing a powerful model. It's about designing the systems around that model so the experience is dependable, intuitive, and genuinely helpful.

Voice-first design requires a different mindset.

Without a traditional chat interface, we had to rethink how users discover features, communicate intent, resolve ambiguity, and move between activities. We learned how important conversational context and interruption handling are to making voice interactions feel natural.

AI works best when paired with deterministic tools.

We learned the value of letting an AI model interpret what a student wants while leaving sensitive or precise operations to structured local systems.

This approach made it easier to reason about PDF retrieval, persistence, scheduling, and navigation.

Grounding matters more than generating more content.

A polished explanation isn't useful if it doesn't match what a student is supposed to learn.

By focusing on exact PDF sections and source verification, we learned how to build an AI experience that prioritizes relevance and traceability.

Good educational experiences are continuous.

Studying isn't a single interaction. Students learn something, practice it, forget parts of it, return later, and gradually improve.

Building Sirius helped us understand why persistent progress, reminders, and contextual learning activities are just as important as AI-generated explanations.

Most importantly, we learned how much more engaging educational software can become when voice, AI, and native device capabilities are designed as one experience.

What's next for Sirius

We see Sirius evolving from a voice-first study companion into a more personalized learning platform.

Some of the directions we're excited to explore include:

🧠 Smarter personalized learning

Using quiz performance and learning history to identify weak areas, recommend review sessions, and adjust question difficulty to each student's progress.

🔁 Spaced repetition

Automatically recommending when students should revisit concepts to improve long-term retention instead of relying on last-minute cramming.

🎧 More expressive audio experiences

Expanding podcast-style lessons with multiple voices, conversational explanations, and different teaching styles.

📊 Better learning insights

Giving students a clearer picture of their progress across classes, chapters, quizzes, and scheduled study sessions.

📚 Expanded study materials

Supporting additional formats beyond PDF textbooks, such as lecture slides, notes, and other course resources.

🌍 Greater accessibility

Continuing to improve hands-free navigation and exploring multilingual lessons and quizzes to make studying accessible to more learners.

🤝 Growing the open-source community

We want other developers, students, and educators to experiment with Sirius, contribute features, and help shape its future.

Ultimately, our vision is simple:

Studying should feel less like managing information and more like having a conversation that helps you grow.

With Sirius, we're building toward a future where every student can have a personalized study companion that listens, teaches, quizzes, and helps them stay on track.

GitHub: https://github.com/sikakwesi2020/Sirius

Built with: Swift, SwiftUI, Gemini Live API, ElevenLabs, PDFKit, Core Data, EventKit, WidgetKit, LiveActivityKit, Keychain, and iOS Notifications.