How to use Google’s PaLM 2 API with LangChain

Large language models (LLMs) have emerged to become an integral part of the software stack. They are available through APIs from providers like Cohere, Google Cloud, and OpenAI or as open-source models hosted on Hugging Face.

However, utilizing LLMs isn’t merely about sending prompts. Developers must consider aspects like parameter adjustments, prompt augmentation, and response moderation. LLMs are stateless, so developers must maintain conversational history for context, possibly using databases for long-term storage.

Furthermore, there isn’t a universal LLM solution. Applications might require various specialized models, complicating integration and increasing development complexity.

LangChain is becoming the tool of choice for developers building production-grade applications powered by LLMs. It has a diverse and vibrant ecosystem that brings various providers under one roof, including Google’s PaLM2 large language model.

In this tutorial, we will walk through the steps of building a LangChain application backed by the Google PaLM 2 model.

Setting up the environment

Visit Google MakerSuite and create an API key for PaLM. 

IDG

In the terminal, create a Python virtual environment and activate it.

python -m venv venv
source venv/bin/activate

Create an environment variable to store the PaLM API key.

export GOOGLE_API_KEY=YOUR_API_KEY

Install the following Python modules.

pip install google-generativeai
pip install langchain
pip install pypdf
pip install jupyter

Accessing the PaLM API

Start a new Jupyter Notebook (on your local workstation or in Google Colab) and run the following code.

import google.generativeai as palm
import os

google_api_key=os.getenv('GOOGLE_API_KEY')
palm.configure(api_key=google_api_key)

prompt = 'Explain the difference between effective and affective with examples'

completion = palm.generate_text(
    model='models/text-bison-001',
    prompt=prompt,
    temperature=0.1
)

print(completion.result)

The program starts by importing the Python modules and then gets the API key from the environment variable. It invokes the generate_text method by setting the model to models/text-bison-001 and passing the prompt variable.

The temperature variable defines the predictability of the model. The values closer to zero will make the output deterministic and predictable.

It generates the following output.

IDG

Because the prompt instructed the model to explain the difference with an example, the model responded with a detailed response.

Now, let’s repeat the same exercise with LangChain.

Start a new Jupyter Notebook and run the code below.

from langchain.embeddings import GooglePalmEmbeddings
from langchain.llms import GooglePalm
import google.generativeai
import os

google_api_key=os.getenv('GOOGLE_API_KEY')

llm = GooglePalm(google_api_key=google_api_key)
llm.temperature = 0.1

prompts = ['Explain the difference between effective and affective with examples']
llm_result = llm._generate(prompts)

print(llm_result.generations[0][0].text)

The code is not only cleaner but simple to understand. It initializes the llm variable by pointing to the Google PaLM model. It then sets the temperature variable.

In LangChain, the prompts parameter is a Python list. You can send multiple prompts at once and get back multiple generations.

Since we only passed one prompt, we accessed the text property of the first generation.

IDG

As we can see, the output is the same as the previous program.

The objective of this tutorial was to introduce PaLM API and the seamless integration with LangChain. The advantage of this approach is the ability to swap the LLM with minimal changes to the code. With LangChain, the LLM becomes one of the “links” of the chain, which can be easily replaced.

In the next part of this series, we will build a LangChain Q&A program based on a custom PDF document. Stay tuned.

This series

• Getting started with Google’s PaLM API
• Build a chatbot with Google’s PaLM API
• Ease into similarity search with Google’s PaLM API
• A brief guide to LangChain for software developers