Gemini API Integration

Overview

This skill guides AI agents through integrating Google Gemini API into applications — from basic text generation to advanced multimodal, function calling, and streaming use cases. It covers the full Gemini SDK lifecycle with production-grade patterns.

When to Use This Skill

• Use when setting up Gemini API for the first time in a Node.js, Python, or browser project
• Use when implementing multimodal inputs (text + image/audio/video)
• Use when adding streaming responses to improve perceived latency
• Use when implementing function calling / tool use with Gemini
• Use when optimizing model selection (Flash vs Pro vs Ultra) for cost and performance
• Use when debugging Gemini API errors, rate limits, or quota issues

Step-by-Step Guide

1. Installation & Setup

Node.js / TypeScript:

npm install @google/generative-ai

Python:

pip install google-generativeai

Set your API key securely:

export GEMINI_API_KEY="your-api-key-here"

2. Basic Text Generation

Node.js:

import { GoogleGenerativeAI } from "@google/generative-ai";

const genAI = new GoogleGenerativeAI(process.env.GEMINI_API_KEY);
const MODEL = process.env.GEMINI_MODEL ?? '<model-id>'; // current model ID — look it up in the provider docs at the time of use
const model = genAI.getGenerativeModel({ model: MODEL });

const result = await model.generateContent("Explain async/await in JavaScript");
console.log(result.response.text());

Python:

import google.generativeai as genai
import os

genai.configure(api_key=os.environ["GEMINI_API_KEY"])
MODEL = os.environ["GEMINI_MODEL"]  # current model ID — look it up in the provider docs at the time of use
model = genai.GenerativeModel(MODEL)

response = model.generate_content("Explain async/await in JavaScript")
print(response.text)

3. Streaming Responses

const result = await model.generateContentStream("Write a detailed blog post about AI");

for await (const chunk of result.stream) {
  process.stdout.write(chunk.text());
}

4. Multimodal Input (Text + Image)

import fs from "fs";

const imageData = fs.readFileSync("screenshot.png");
const imagePart = {
  inlineData: {
    data: imageData.toString("base64"),
    mimeType: "image/png",
  },
};

const result = await model.generateContent(["Describe this image:", imagePart]);
console.log(result.response.text());

5. Function Calling / Tool Use

const tools = [{
  functionDeclarations: [{
    name: "get_weather",
    description: "Get current weather for a city",
    parameters: {
      type: "OBJECT",
      properties: {
        city: { type: "STRING", description: "City name" },
      },
      required: ["city"],
    },
  }],
}];

const model = genAI.getGenerativeModel({ model: process.env.GEMINI_MODEL ?? '<model-id>', tools });
const result = await model.generateContent("What's the weather in Mumbai?");

const call = result.response.functionCalls()?.[0];
if (call) {
  // Execute the actual function
  const weatherData = await getWeather(call.args.city);
  // Send result back to model
}

6. Multi-turn Chat

const chat = model.startChat({
  history: [
    { role: "user", parts: [{ text: "You are a helpful coding assistant." }] },
    { role: "model", parts: [{ text: "Sure! I'm ready to help with code." }] },
  ],
});

const response = await chat.sendMessage("How do I reverse a string in Python?");
console.log(response.response.text());

7. Model Selection Guide

Never hard-code a model name from memory — model IDs change frequently. Resolve one variable (GEMINI_MODEL, same convention as the gemini-api-dev skill) from the environment and set it to the current ID from the official models page: https://ai.google.dev/gemini-api/docs/models — look it up at the time of use. Pick among three distinct capability tiers first, then take today's ID of that tier from the page:

Tier	Best For	Speed	Cost
Fast	Simple classification, short completions, high volume	Very fast	Lowest
Balanced	Everyday multimodal calls, function calling	Fast	Low
Reasoning	Complex reasoning, long context, agentic work	Slower	Higher

The tier only decides which current ID you assign to GEMINI_MODEL; the tier names themselves are never valid --model/API values.

Best Practices

• ✅ Do: Use the current fast-tier model for most tasks — it's typically the cheapest and quickest option in the catalogue
• ✅ Do: Always stream responses for user-facing chat UIs to reduce perceived latency
• ✅ Do: Store API keys in environment variables, never hard-code them
• ✅ Do: Implement exponential backoff for rate limit (429) errors
• ✅ Do: Use systemInstruction to set persistent model behavior
• ❌ Don't: Use the most capable (premium-tier) model for simple tasks — fast-tier models are cheaper and quicker
• ❌ Don't: Send large base64 images inline for files > 20MB — use File API instead
• ❌ Don't: Ignore safety ratings in responses for production apps

Error Handling

try {
  const result = await model.generateContent(prompt);
  return result.response.text();
} catch (error) {
  if (error.status === 429) {
    // Rate limited — wait and retry with exponential backoff
    await new Promise(r => setTimeout(r, 2 ** retryCount * 1000));
  } else if (error.status === 400) {
    // Invalid request — check prompt or parameters
    console.error("Invalid request:", error.message);
  } else {
    throw error;
  }
}

Troubleshooting

Problem: API_KEY_INVALID error Solution: Ensure GEMINI_API_KEY environment variable is set and the key is active in Google AI Studio.

Problem: Response blocked by safety filters Solution: Check result.response.promptFeedback.blockReason and adjust your prompt or safety settings.

Problem: Slow response times Solution: Switch to the current fast-tier model and enable streaming. Consider caching repeated prompts.

Problem: RESOURCE_EXHAUSTED (quota exceeded) Solution: Check your quota in Google Cloud Console. Implement request queuing and exponential backoff.

Limitations

• Use this skill only when the task clearly matches the scope described above.
• Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
• Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.