yogpt: Using ChatGPT and Other LLMs from the Command Line

Recently, we tend to use chat language models more and more in our everyday activities. If you are a traditional office worker, using it through web chat interface or co-pilot seems natural. However, for developers, who spend a lot of their time on the command line, having terminal interface for chat models is totally useful.

There are already some projects that give command-line interface for ChatGPT, for example, heygpt. However, it only gives you access to OpenAI ChatGPT model, while I wanted to have access to other models, such as Yandex GPT and GigaChat. Moreover, heygpt is implemented in Rust, which makes it pain to install for those of us who live in Python ecosystem.

Welcome yogpt

This forced me to create my own command-line GPT tool called yogpt over the weekend. Here are the main principles that I followed:

• The tool should be pip-installable. Most people nowadays would have Python command-line on their computer, and running pip install yogpt would make the tool immediately available.
• It should support any chat model supported by LangChain - making a plethora of models readily available, including original ChatGPT, GigaChat, YandexGPT, etc.

There are a few ways yogpt can be used:

• By asking the question to GPT directly on the command line, for example:

  $ yogpt What is the 10th digit of Pi?
  

• By piping stdin into yogpt. If yogpt understands that it is invoked in the pipe, it will automatically take pipe as input. You can also explicitly specify - in the command-line, for example:

  $ echo What is the 10th digit of Pi | yogpt -
  

• Calling yogpt without input initiates console chat, and you can talk to the model interactively.
• Sometimes it makes sense to further chat with yogpt after providing it with some input. For example, you can ask it to read Python program, and then answer your questions about it. In this case you may specify -c/--chat flag. In this case it will consume input as first utterance to the bot, and then initiate a follow-up chat, in which you can ask for more details. For example, you can ask GPT to analyze a Python program (this one):

$ yogpt -s "You are a software expert. Please read python program
  provided and be ready to answer questions on this program."
  -c @game.py

This program is a simple implementation of the game Space Invaders
using the Pygame library in Python. It includes a player-controlled 
spaceship that can move left and right, shoot bullets, and destroy 
enemy spaceships. The player earns points for each enemy spaceship 
destroyed.

 U> How is collision detection implemented?
AI> Collision detection in this program is implemented using the 
    `groupcollide` function provided by the Pygame library. The 
    `groupcollide` function is used to detect collisions between 
    two sprite groups and manage the sprites that collide.
 U> ^C

The example above also shows that you can use @filename syntax to get input from a file. Also, we have specified system prompt here via -s parameter, instructing the model what it should do.

System Prompt and Prompt Templates

In many cases, we want to instruct the model to do something to the data that we provide to it as part of the prompt. For example, if we want to use GPT to translate a program from one programming language to another, we need to provide those instructions. This can be done in two ways:

• By specifying system prompt via -s/--system parameter:

$ echo print([2**i for i in range(10)]) | 
  yogpt -s "Translate the program that you are given as input to C++"

Here is the result you are likely to get:

#include <iostream>
#include <vector>
#include <cmath>

int main() {
    std::vector<int> result;
    for (int i = 0; i < 10; i++) {
        result.push_back(pow(2, i));
    }
    for (int i = 0; i < result.size(); i++) {
        std::cout << result[i] << " ";
    }
    return 0;
}

• By using prompt template via -p/--template parameter:

$ echo print([2**i for i in range(10)]) | 
  yogpt -p "Translate the program that you are given below in 
  double square brackets to {param_1}:\n[[{}]]. Provide only 
  program text as output." -1 Javascript

This gives the following result:

console.log(Array.from({length: 10}, (_, i) => 2 ** i));

You can see that prompt template provides more flexibility, because you can use different prompt engineering techniques such as delimiting the input explicitly, and fine-tuning the prompt (in our case I have added Provide only program text as output to avoid getting some additional explanation on what the program does).

Also, I have used additional parameter here, which specifies destination programming language. By using constructions from {param_1} to {param_3} in the template, I can then specify those parameters by -1 to -3 command options. This is especially useful if I am getting a template from a file or config, and cannot easily adjust it on the command line.

For both system prompt and prompt template, we can specify @filename in order to read prompt from file. However, even better way is to put prompt templates that you often use into config file, as described below.

Installation and Configuration

Now that you have seen a few examples of using yogpt, you are probably wondering how to start using it.

As I mentioned, installing the model on your local computer, provided that you have Python 3.10+ tooling available, can be done using:

$ pip install yogpt

It is important to have a version of Python>=3.10, because that is what LangChain requires.

If you just do pip-install without any configuration, the tool would try to use GPT4Free project to find some way to access ChatGPT model online. However, I do not recommend doing so in the long run - please spend some time configuring your own list of models and your own credentials.

Configuration is done by creating a .yogpt.config.json file in your home directory (which is typically c:\users\<username> on Windows, or /home/<username> on Linux). This is a JSON file that allows you to specify models you want to use, including your personal API keys. For example, here is the file that specifies Yandex GPT and GigaChat models:

{
 "models" : [
   {
     "name" : "yagpt",
     "classname" : "langchain.chat_models.ChatYandexGPT",
     "default" : true,
     "params" : { "api_key" : "..." }
   },
   {
     "name" : "gc",
     "classname" : "langchain.chat_models.GigaChat",
     "params" : { "credentials" : "... ", 
                         "verify_ssl_certs" : false }
   }
]}

You can check out sample .yogpt.config.json file in the project repository here and use it as a starting point.

As you can see, for each model we supply class name of the model class (which can be part LangChain, but you can also specify other compatible models), as well as parameters that we pass to the class constructor. We can also mark any model in the config file as default.

Having defined your models in config, you can specify the model when calling yogpt like this:

$ yogpt -m yagpt Привет, расскажи анекдот про число пи!
Почему число Пи такое большое? Потому что оно округляет!

(The text here is in Russian, because Yandex GPT model works best with this language)

Also, in the same config file you can define prompt templates and system prompts that you often use. Suppose you often want to use yogpt to translate text into different languages, in this case you can define the following template in templates section:

{
  "name" : "translate",
  "template" : "Please, translate the text in double square brackets
    below into the {param_1} language. Here is the text:\n[[{}]]"
}

In which case translation would be done using the command:

$ echo How do you do? | yogpt -p translate -1 german -t 0.01
Wie geht es Ihnen?

Here I have specified the --temperature or -t parameter, in order to get less hallucinations in the output (i.e. more accurate translation). If I want the model to be more creative, I would probably specify higher value (from 0 to 1):

$ yogpt -t 0.6 Invent 5 funny names for a cat of Python developer.
1. Bytey Whiskers
2. Code Paws
3. Monty Purthon
4. Snaketail Scriptmeow
5. PyFluff Constrictor

Summarizing blog posts

In the text about, you have seen a few examples of using yogpt in tasks from understanding computer programs to translation. There are many possible example of command-line usage for GPT, from translating LaTeX formula to Python, to re-writing blog posts in a different tone of voice. For example, this blog post contains a lot of nice examples of using GPT in mathematician’s everyday life, and most of them are from command line.

Let’s stick to the latter example of processing blog posts. For some not-so-long blog posts (which are short enough to fit into context window of current language models), I try to use yogpt for translation. I have the following prompt blogtran defined in my config file: I want to translate my blog post written in Jekyll markdown format into Russian. Please leave the configuration section intact, and translate the rest of the text, leaving markdown syntax. The blog post is provided below: {}, which allows me to use:

$ yogpt -p blogtran @blog.md > blog_en.md

It is known that GPTs provide better translation than specialized translation systems, but they have shorter context, and sometimes can hallucinate.

Much better use case for GPTs are text summarization, which allows us to extract most important information from the blog post. The prompt I may use for summarization is the following: Please, read the post below in Jekyll Markdown and write its short summary. Output results in JSON format with “title” and “summary” fields. Here is the post:{}. I will put it into summarize.txt, and then loop over blog posts to extract their summary:

for %c in (*.md) do; 
 yogpt -m yagpt -p @summarize.txt @%c >> result.json

This will create result.json file, which will not exactly be well-formed JSON, but will look similar:

{
  "title": "Neurogenerative Models and the Future of Art",
  "summary": "The post discusses the latest developments in neurogenerative models, focusing on Stable Diffusion. It explores the capabilities of text-to-image models, showcasing examples and speculating on their impact on the art world. The author highlights the release of Stable Diffusion's code and weights, emphasizing its significance for artists. The post also delves into prompt engineering, creative uses of neural generation, and the potential for AI to enhance the artistic process. It concludes with new ways to experience neural art, including neurogenerative streams, integration into art objects, and the concept of neurogenerative parties."
}
` ``json
{
  "title": "Never Trust a Neural Network!",
  "summary": "The post highlights the untrustworthiness of ChatGPT and other neural network models for providing accurate information. It illustrates this point by requesting absurd information from ChatGPT, such as reasons why eating cucumbers leads to sleeping disorders or why learning mathematics can be deadly. The post concludes with a mix of accurate and inaccurate historical information generated by ChatGPT, emphasizing the model's limitations and the need for skepticism."
}
`` `
{
  "title": "Creating Domain-Oriented Chatbots using LangChain and Yandex GPT",
  "summary": "The article discusses the creation of domain-specific chatbots using the Retrieval-Augmented Generation approach with LangChain and Yandex GPT. It covers steps such as converting video to text, breaking text into fragments, calculating embeddings, saving documents to a vector database, implementing retrieval-augmented generation, and context transformation. It emphasizes the use of LangChain for various tasks and demonstrates how to build a question-and-answer bot based on a video collection."
}

In some of the cases this malformed JSON includes markdown formatting elements, and it lacks grouping of all individual JSON objects into one list. However, it does not matter if all we want to do is further process this with GPT, because LLM can handle slight deviations from JSON format. For example, to render this into HTML table, we can just say:

$ yogpt -p "Please take a list of JSON objects with title and
  summary fields and represent them as HTML table"
  @result.json > result.html

And we will get HTML that can be used to display the summary:

Of course, we can also ask GPT to represent this as markdown table, or as sequence of SQL statements to insert the data into the database.

Of course, instead of writing a blog summary, I can ask GPT to produce a selling text for it, which I can then put on the front page as a banner. I just need to adjust the prompt a little bit: Please, read the post below in Jekyll Markdown and write a short motivational text, telling people why should they read it. Output results in JSON format with “title” and “summary” fields. Output pure json, without any decorations. Here is the post:{}. After repeating all the steps above with the new prompt, the result will be:

Keep in mind that those HTML tables, including formatting, were created automatically from original blog posts using just two command lines in a matter of minutes!

Takeaway

I think the examples shown above demonstrate the true transformative power of GPT models, which you can now freely use from the command-line! In case you like yogpt, fell free to leave your start, comments and suggestions on the GitHub page. And if you have some ideas on how to improve the tool - the best way to do it would be through a pull request, because I do not have much time to spend on improving this tool myself, but would welcome any help!

Dialogue & Discussion