Lightning Model

Just pay for what you use with Bitcoin Lightning—whether that's per token or per request.

Comment

Inspiration

The inspiration for this project came when I was learning about how AI generate content and give precise output. Then I found out about the tokens and limitations fixed on usage by companies. For pay per token where user has the ability to decide to what extent they want to use and just pay for that. Lightning Network offers just what I trying to implement: Micropayments.

What it does

Lightning Model provides on-demand access to Google's Gemini AI model through Bitcoin Lightning payments. Users can choose between two flexible pricing options:

Pay-per-token: Users pay only for the exact number of tokens consumed, making it cost-effective for variable usage patterns and enabling flexibility for custom token limit selection.

Pay-per-request: Users pay a fixed amount per API call, offering predictability for those who prefer simpler pricing.

After selecting a payment model and making a small Lightning payment, users get immediate access to chat with the AI. The platform tracks usage in real-time, showing remaining tokens or requests. When users reach their limit or finish their work, the session terminates—no subscriptions to cancel, no recurring charges to worry about.

Tokens are basic units of text that model processes. Eg. Hello World! Tokens: [Hello], [World], [!]

How we built it

Backend: Python with FastAPI provides a robust API layer. We integrated Google's Generative AI library to access Gemini's capabilities and implemented SQLAlchemy for data persistence.

Rate Limiting: We developed a Redis-based token bucket algorithm to manage usage fairly and prevent abuse.

Payment Processing: We integrated with Lightning Network through a custom LN node deployed on voltage for secure, instant micropayments.

Frontend: Vite+React+Javascript+TailwindCSS for user interface.

Challenges we ran into

1.Handling streaming responses efficiently: Initially, we struggled with implementing streaming responses from Gemini while accurately tracking token usage in real-time. This required building a custom token counting solution that works alongside the streaming functionality.

2.To find and implement appropriate methods to count tokens such as usage_metadata.

Accomplishments that we're proud of

  1. Learned a new blockchain technology- Lightning Network

  2. Creating and handling different types of Lightning nodes and implementing them in our project

  3. Learned different methods such as Token Bucket Algorithm and experimented with API Response Headers

  4. Handling new Databases like Redis and SQLAlchemy

What we learned

We learned a lot during this project where we were first introduced to different technologies associated with Bitcoin such as Lightning. We have started reading Mastering Lightning Book for an depth knowledge of these payment systems. Also this project improved our web page styling a lot.

What's next for Lightning Model

1.Multi-model support: Adding access to a wider range of AI models beyond Gemini.

2.Usage analytics: Providing detailed insights into token and request patterns.

3.Multi-model routing: Building an intelligent routing system that can dispatch requests to different models based on the query type, optimizing for both cost and performance.

Built With

Share this project:

Updates