Using GPT for natural language querying

Rafal Cymerys

July 19, 2024

Night mode

Year 2023 was supposed to be the year of AI. For sure we all see some changes, especially in social media walls. For example, my LinkedIn is flooded with AI generated texts or images.
However, I prefer to focus on the realms of Artificial Intelligence that are equally fascinating and beneficial. I tried using AI, specifically GPT, to transform natural language into structured, machine-readable forms. This approach is used to generate machine-readable forms that translate into other use cases as well, i.e. the insights in this blog post can inform other equally interesting and useful applications. In this article you will learn how to use GPT for data processing and allowing users to provide natural input to an application. Let’s take a look into some interesting use cases.

It turns out that GPT does a pretty good job of parsing natural language input into data structures defined by a schema. We can use it for transforming manuals into a JSON graph, that can be then used for, e.g. presenting step-by-step instructions or visualizing dependencies.

Example: Let’s cook!
We can use GPT to transform a recipe into a graph, defined by a JSON structure. It can then be used to, for example, present step-by-step instructions or visualizing dependencies between them.

I found a pretty good spaghetti recipe available here.
We can try transforming it into a JSON.

First, let’s define a JSON schema of the JSON that we’d like to get from GPT:

The schema can look like this:

‍
Then a prompt for GPT. We’ll split it into three parts:
1. Schema definition
2. The recipe to be parsed
3. A summary of what we expect it to return (just the JSON, conforming to the schema)

‍
To get stable results, you may need to tweak the prompt. We’ve noticed that more complex domains GPT will often require a more complex prompt that more explicitly specifies our requirements. For a spaghetti recipe it’s enough though.
Sample output:

‍
This is something we can easily display on a graph:

‍
If we aren’t satisfied with the outcome, we can also use a follow-up prompt to mutate the structure. If you look at the recipe, step 8 (cook spaghetti) is independent of the previous steps.

Let’s ask GPT to change that:

As the previous response is a part of the follow-up prompt, GPT does a pretty good job at keeping the unchanged elements stable.

Use case - build a BPMN specification of a business process

Let’s expand this further - this approach allows us to generate a BPMN definition of a business process, which can subsequently be used by a tool like Camunda.

Let’s try the following prompt:

Where process description could be for example:

After sending that to GPT, we’ll get the following output:

Looking good!

Remarkable insights:
1. GPT seems to omit some of the XMLNS references quite often. This can be easily fixed in post-processing, which may be way more cost-effective than regenerating the response.
2. I once got a response where the diagram definition looked like the following:

Even though it’s quite amusing, this is an edge case that we’d likely need to support if aiming for production use. The most common approaches are to add more validations and retries to our application’s code, improve the prompt to direct GPT, or try different temperatures of the model.

Use case - build dynamic forms in React

Another common use case that we can try to automate is building custom forms dynamically. Many applications, such as surveys, custom forms etc., rely on this feature to gain feedback from its candidates, customers or potential leads.

GPT makes it incredibly easy to build a tool that allows a regular user to create such a form by providing natural language input. Let’s use a library called react-jsonschema-form. It’s a React-based library that generates forms based on a JSON definition. It also contains an extensive schema definition that such JSONs can be validated against.

We can leverage the fact that GPT was trained on a large dataset that also includes definitions of commonly used JSON schemas. That means we don’t have to provide the exact schema, and we can refer to it in our prompt by name.

Let’s create a prompt that:
1. Specifies that we would like to use the schema of react-jsonschema-form
2. Include the user's definition of the ideal form
3. Will specify some additional criteria for the output (i.e. make sure that we only get a JSON as a response)

Where userInput could be:

In this case GPT gets back to us with

We can quickly verify it in react-jsonschema-form’s playground: https://rjsf-team.github.io/react-jsonschema-form/

As you can see, it correctly generated a form, together with relevant validations. As with previous examples, we can make modifications by follow-up prompts.

Let’s try making the privacy policy field selectable by default. Sending the following prompt:

Will result in a new JSON definition of the form:

Which looks like that:

As you can see, the checkbox is now selected by default. Similar to that, we can make further tweaks to the existing form.

Use case - build dynamic UIs

In the previous example, we were bound to forms, which are useful but limited in terms of what we can do. To build more general UIs dynamically, we can use a tool called react-json-schema, combine it with GPT, and build fully dynamic complex UIs.

To start with, let’s define some components. We’ll use react-bootstrap to simplify the UI work needed.

Let’s make a wrapper component for the whole app:

Then a basic layout component:

A customizable navbar:

A toolbar with some action buttons (that we can wire together later):

And a table for displaying a list of users:

Then connect it all together in our web application:

It’s easy:
1. Initialize react-json-schema parser
2. Configure it to recognize our custom components
3. Prompt GPT for a schema based on user’s description
4. Render the schema that GPT responded with

Now it’s time to define the actual prompt to be used:

Remarkable insights: As you can see, we started with a list of available components, followed by an additional list of props available for each of them. My tests have shown that it’s also helpful to explicitly state how to provide props (without a nested props object). Without that remark, GPT put it inside a nested object, which didn’t work with react-json-schema library. My assumption is that react-json-schema is not a very popular library and lacks extensive examples online - hence the need to point GPT in the right direction.

A sample GPT response will look like:

PICS Which renders a quite decent dashboard! GPT was able to infer that the majority of the layout should go inside a component called AppMainLayout, but didn’t put the AppNavbar there. It also filled the navbar with some example links.

## Use case - querying an ElasticSearch index

ElasticSearch is commonly used for building indexes of products or application logs. We can use GPT to translate user’s queries to an ElasticSearch query.

The approach would be:
1. Specify that we’d like to get an ElasticSearch query as a result
2. Provide information about the structure of the index
3. Provide information about the user's query
4. Specify additional criteria for the output

Example: We have an ElasticSearch index with the following definition:

It’s worth mentioning that this approach should never be used on an ElasticSearch that’s not meant to be fully accessible by the user due to the potential risk of making arbitrary queries to the cluster.
It comes back with the desired query:

We can now pass it to ElasticSearch and get the results:

Follow-up use case - building a query in a query language that the user may not know

This feature is useful when a user needs to query a tool with its own specific query language or with a structure that the user doesn’t know. AWS seems to be running an A/B test of a feature where you can generate a CloudWatch search query based on a natural language input.

## Use case - data visualization

Interesting use case - you allow users to pull report data (by e.g. querying an ElasticSearch index or a database). This is often combined with visualizing the data. Could we allow the user to also define how they’d like the data presented? Let’s say a manager needs a pie chart for their PowerPoint presentation.

Fortunately, there’s a tool called Vega which is a visualization grammar that allows you to define graphs using a JSON schema. It also comes with Vega-Lite, which allows us to easily render these charts using JavaScript. There’s also an interactive playground that we can use for testing our diagrams.

Let’s take a sample CSV file from Vega-Lite’s demo that contains the stock prices history of a few tech companies. It will be easy for us to reference it on the playground.

The file is a regular CSV that contains a few columns:

Let’s try to generate some charts that utilize this structure.

After trying it out with GPT we get the following definition

Which gives us the desired set of charts

There’s also an interesting project on Github that showcases a similar concept. Their base prompt looks slightly differently:

Good practices - learn from my experiences

Once the request was submitted to ChatGPT, we should run a JSON schema (or XML) validator on the response it provides. Despite its reliability, it may act differently than expected. Sometimes a retry will be enough to get a result that’s consistent with the defined structure. On the user experience side, there are several approaches, but you should tailor them to your specific needs. A good example of JSON schema validation is using Vega-Lite’s JSON Schema, available here. You can apply the same method with any JSON schema you create, like for a spaghetti recipe.

Complexity - try not to bite more than GPT can chew‍

The prompts can’t be too long, as GPT may lose information and context with complicated inputs. You can use clever prompting engineering to break a task into smaller ones that gradually reduces the cognitive load and follows a logical reasoning process. For example, instead of asking GPT to format a recipe in a specific JSON schema, at first break it down into steps “Break this recipe down into steps". Once that’s achieved - "Structure this as the following JSON schema". It’s about breaking it down into steps, where each step builds on the understanding of the previous one and aligns it with the defined structure.

Performance - being patient is the key‍

Generating outputs with GPT is quite time-consuming, especially if we need to process multiple prompts in order to achieve the desired use case. We can expect improvements when it comes to hardware, so this won’t always be an issue, but for now, we need to focus on the UX level to cover up these issues. Improving UX in such cases is challenging, and often the best solution is to stream the response in smaller parts. However, if the model needs to return a full result (e.g. a JSON schema), this approach doesn’t work.

Cost - manage your expenses

However, using e.g. GPT-4 can be quite costly at scale. I wouldn’t be too worried about that in the long term, as like any technology, it will get cheaper over time, but for now it’s good to keep an eye on how much money is spent. The most sensible approach is to perform as much preprocessing as possible before pushing input to GPT, in order to limit the cost of the actual GPT usage.

Reliability - fine-tuning for diverse inputs

Playing around with these examples makes it quite easy to get it to a point where it fairly consistently delivers a great result. However, once you launch a GPT-based feature to a wider audience that will use it with different input data, you may need plenty of fine-tuning in order to ensure the solution's reliability. You never really know what the user’s will type into the chat box once you deploy your app. As developers of the feature, we are usually biased. User testing those solutions is important, especially since using a chat interface to solve actual problems (rather than just getting answers) is still a pretty novel idea and people have different expectations.

Prompt injection risks - always double check

It goes without saying that you need to be careful about the responses it may generate // (Optional) Screenshot of DPD example (the one where a customer prompted it to write a song about a shitty delivery company) ![[dpd_chatbot_fail-1.png]]

Summary

GPT does a pretty good job in parsing data. Basing on the experiments described in the post, we can define a simple framework that can handle diverse input and output formats:
1. Specify the desired schema
2. Specify the details of available fields (when needed)
3. Specify the input data
4. Reinforce important details (e.g. that the response should be a JSON and only a JSON)

Even now, there are many applications that can utilize GPT, without being AI-first. It doesn’t need to be a whole GPT-based experience, it can supplement the existing experiences and improve user productivity.

‍

Back to blog