Building an Offline AI Learning Companion: My Journey with Gemma

In the vast landscape of AI, where models often feel like distant, complex entities, I stumbled upon something different — Gemma. Not just another large language model, but a companion that resonated with my aspirations, challenges, and the desire for accessibility in AI.

My Journey with Gemma

In the ever-evolving world of AI, I embarked on a mission to create a personal learning assistant for students — a tool that could summarise academic PDFs and generate quizzes to aid their studies. However, during my research and surveys, I discovered a significant hurdle: many students lacked consistent internet connectivity. This realisation led me to seek a solution that could operate offline, ensuring accessibility for all.

🌐 The Challenge: Bridging the Connectivity Gap

The initial versions of my AI tool relied heavily on cloud-based models, which necessitated a stable internet connection. This dependency contradicted my goal of creating an inclusive educational resource. I needed a solution that could function seamlessly without internet access, especially in regions where connectivity is unreliable.

🔍 The Discovery: Unveiling Gemma

My quest led me to Gemma, Google’s open-source, lightweight large language model (LLM). Designed with efficiency in mind, Gemma can run directly on devices, eliminating the need for constant internet connectivity. Its adaptability and open-weight nature made it the perfect candidate for my project.

Implementing Gemma: A Game-Changer

Integrating Gemma into my application transformed the user experience. Students could now upload their academic materials and receive summaries and quizzes without needing to be online. This offline capability ensured that learning was uninterrupted, regardless of internet availability.

Moreover, Gemma’s compatibility with tools like Hugging Face and its support for fine-tuning allowed me to tailor the model to the specific needs of my users. By training it on educational datasets, I enhanced its ability to generate relevant and accurate content.

Why Gemma Resonated with Me

Runs Locally: The ability to operate Gemma offline was a game-changer. Whether on a train or in a remote area, I could interact with the model without worrying about connectivity.

Customizable: Gemma isn’t a one-size-fits-all. I could fine-tune it to understand and generate content specific to my domain, making it feel like a personalized assistant.

Resource-Friendly: Unlike other models that require hefty GPUs, Gemma ran smoothly on my modest setup, proving that powerful AI doesn’t have to be resource-intensive.

Empowering Mobile Learning

One of the most remarkable aspects of Gemma is its ability to run on mobile devices. Utilizing the MediaPipe LLM Inference API, I deployed Gemma on smartphones, making the learning assistant truly portable and accessible . Students could now engage with the AI tool anytime, anywhere, without worrying about data usage or connectivity issues.

Reflecting on the Impact

The integration of Gemma into my project not only solved the connectivity challenge but also opened new avenues for personalized and accessible education. It reinforced my belief in the power of open-source tools to drive innovation and inclusivity.

🔗 Resources for Fellow Innovators

If you’re inspired to explore Gemma for your projects, here are some resources to get you started:

Official Documentation: Google AI — Gemma

Mobile Deployment Guide: Deploy Gemma on Mobile Devices

Fine-Tuning Tutorial: Fine-Tune Gemma Using Hugging Face

Community Examples: Gemma Cookbook on GitHub

Gemma’s Capabilities: A Closer Look

1. Versatility Across Tasks

Gemma excels in various natural language processing tasks, including:

Text Summarization: Condensing lengthy academic materials into concise summaries.

Question Answering: Providing accurate responses to user queries.

Language Translation: Facilitating multilingual communication across over 140 languages .

2. Multimodal Processing

With the release of Gemma 3, the model has expanded its capabilities to process not just text but also images and short videos. This multimodal functionality opens doors to innovative applications in education, content creation, and more .

3. Efficient Performance

Designed to run on devices with limited computational resources, Gemma delivers impressive performance without the need for high-end hardware. Whether it’s a single GPU, a mobile device, or even a CPU, Gemma ensures that AI remains within reach for all

To use Google’s Gemma models on Kaggle, follow these steps:

Step 1: Obtain Kaggle API Credentials

Log in to your Kaggle account.

1. Navigate to your Account settings.
2. Scroll down to the API section and click on “Create New API Token”.
3. This will download a kaggle.json file containing your API credentials.

Step 2: Upload kaggle.json to Kaggle Notebook

Open a new Kaggle Notebook.

1. In the notebook interface, click on the “Add-ons” menu and select “Secrets”.
2. Click on “Upload” and upload the kaggle.json file you downloaded earlier.
3. Once uploaded, Kaggle will automatically set the KAGGLE_USERNAME and KAGGLE_KEY environment variables for you.

Step 3: Set Up the Environment

In your Kaggle Notebook, install the necessary libraries:

!pip install -q tensorflow-cpu
!pip install -q -U keras-nlp tensorflow-hub
!pip install -q -U "keras>=3"
!pip install -q -U tensorflow-text

Then, configure Keras to use the JAX backend for optimal performance:

import os

os.environ["KERAS_BACKEND"] = "jax"
os.environ["XLA_PYTHON_CLIENT_MEM_FRACTION"] = "1.0"

Step 4: Load the Gemma Model

Import the necessary modules and load the Gemma model:

import keras
import keras_nlp

# For reproducibility
keras.utils.set_random_seed(42)

# Load the Gemma model
gemma_lm = keras_nlp.models.GemmaCausalLM.from_preset("gemma2_instruct_2b_en")

You can view the model summary to understand its architecture:

gemma_lm.summary()

Step 5: Generate Text with the Model

To generate text using the model:

prompt = "Explain the significance of the moon landing in 1969."
response = gemma_lm.generate(prompt, max_length=100)
print(response)

This will output a generated text based on the provided prompt

Click here to see Colab

Google Colab

Note: In my next blog we will explore some projects using gemma

Building an Offline AI Learning Companion: My Journey with Gemma was originally published in Google Developer Experts on Medium, where people are continuing the conversation by highlighting and responding to this story.