Deep dive into the APEX AI components

Why would you care?

By seeing how the the AI features work, you will be able to understand better the whole process of using Large Language Models in APEX. You'll see the connections between settings in the Builder and the actual requests to the LLM APIs. That should give you an idea what the benefits of the current approach are and what possible improvements can be made.

APEX in the AI era

As described in my previous blog post about the new AI features in APEX 24.1, there are a number of places throughout the Builder and user facing applications, where they are applicable:

🔸 AI-Assisted App Development
🔸 AI-Assisted SQL Authoring
🔸 AI-Assisted Debugging
🔸 Create Apps using Natural Language
🔸 Conversational AI Dialogs
🔸 APEX_AI API

https://blog.apexapplab.dev/apex-in-the-ai-era

APEX Generative AI Services configuration

💡

Just a quick refresher - We configure the access to LLMs that are used in APEX, by going to Workspace Utilities / Generative AI. Screenshots and more details in my previous blog post (link above) ⤴️

But here is something that has not been previously discussed, regarding the Generative AI Service configuration - the Additional Attributes option. This option allows you to include additional attributes that will be added to the JSON request payload to the LLM REST API. Such attributes include response_format, temperature, top_p, tools (functions calling) and so on. Full list can be found at OpenAI's API documentation:

https://platform.openai.com/docs/api-reference/chat

Here is an example of using the response_format attribute and how that influences the responses when used in the AI Assistant component in APEX.
1️⃣ The first example is the default one - no additional attributes added.
2️⃣ In the second one, response_format is specified to be a JSON Object.

Once the settings are saved, I did a test with each of these configurations, using the AI Assistant chatbot. Here are the results with each of them.

1️⃣ In the case where no additional attributes were specified, the response from the LLM was in plain text, which is useful when you are developing a conversational feature like chat or Q&A bots.

2️⃣ In the second case, you can see the additional attribute being added to the REST API call payload. Specifying json_object as response_format, forces the LLM to return its responses as JSON. This gets handy if you use it for some specific features that work with JSON. It is also easier to use JSON inside SQL and PL/SQL so another benefit of this attribute, especially in the context of APEX development.

Deep dive into AI Assistant component

💡 Having discussed the configuration of AI Services and the effects Additional Attributes might have, it's time to dive deeper into Prompt Engineering, trace and explain the REST API requests that go in and out of different APEX AI components.

This is the new AI Assistant in action. On the graphics above, you can see the whole process - setting up the Dynamic Action that renders the Chat window, mapping of each setting and the REST API Request to the LLM with the System and User Prompts included.

1️⃣ This is a Static Item on our page that we are going to use for the Initial User prompt. Similarly, we could use a Hidden Item that gets populated with different information if your functionality requires it.

💡

To make a good use of the LLM, you can display your relational data as a big text block or a JSON structured document. In the example above, I have combined information from multiple tables and columns into a readable text with distinct sections for Patient Information, Medical History, etc.

2️⃣ A Dynamic Action of the new Open AI Assistant type. Possible configuration options include:

🔸 Service - pick from the list of the Generative AI Services already configured.
🔸 System Prompt - System prompts can be used to inform the LLM about the context. The context may be the type of conversation it is engaging in, or the function it is supposed to perform. In general it helps the LLM generate more relevant, accurate and appropriate responses. And the better the System prompt is - the better the LLM responses to User prompts will be. There is only one system prompt in the REST Request and it is evaluated first, before all other user messages.
🔸 Welcome Message - This is just an informative message that is displayed as first message from the AI Assistant in the Chat window. It is not included in the REST Requests to the LLMs.
🔸 Appearance - Specify a title for the Chat region, as well as the way it is displayed - Inline or as a Dialog.
🔸 Initial Prompt - Specify additional User messages to be used when calling the LLM. You can specify a Page Item (or Javascript expression) and include a big chunk of text, like the patient record in the example below. In many cases this might be a very long text. By default it will be displayed in the Chat.

💡

If you want to display a shorter message instead of the full text of the selected Item, fill the Displayed Message property. In this case the short text will be displayed in the Chat, but the full text will be used when calling the LLM service. Look at the examples below to see how Displayed Message appears on top of the conversation.

🔸 Immediate Action Prompt - Whatever you enter there will be sent as a User message as soon as the Chat is opened.
🔸 Use Response - Allows you to use the Chatbot (LLM) response and assign it back to an Item on the page. You can optionally specify a label for the Button, if not - the default will be "Use this".

3️⃣ AI Assistant Chat window. It is generated using the new ai.js and da.ai.js Javascript files. It has all the needed classes and methods to set the configuration, render the Chat region and deal with AJAX calls to the LLM Service you have configured. It is also used to handle User consent for using an LLM Service, chat icons, etc.

4️⃣ This is the Body of the REST Request sent to the LLM provider. As you can see, each user message is added as "role": "user" JSON element, while the responses appear as "role": "assistant" content. Each time you add a new message, the REST request adds all previous "user" and "assistant" messages (the chat history) too.

💡

APEX keeps the chat context only while the Chat windows is open. Once you close it, the chat history is lost and context cleared. The next time you open the chat window, the AI Assistant will not be aware of your previous conversations.

💡

Each message you send in the chat triggers a new REST request to the LLM. The body of the request appends the last User message and LLM response. This is how the context is preserved during the chat session and the LLM is aware of the whole conversation. This also means that with each new message, the number of tokens used gets bigger and bigger - something that will affect the total number of tokens used and your bill if you are using a paid service provider like OpenAI or Cohere.

{
    "model": "openhermes2.5-mistral:latest",
    "messages": [
        {
            "role": "system",
            "content": "You are a helpful medical assistant."
        },
        {
            "role": "user",
            "content": "###\n\\# Patient Record: John Doe\n\n\\# Personal Information:\n\nName: John Doe\nDate of Birth: January 15, 1980\nGender: Male\nAddress: 1234 Elm Street, Springfield, IL 62704\nPhone: (555) 123-4567\nEmail: johndoe@example.com\n\n\\# Medical History:\n\n- Visit Date: March 15, 2023\nReason for Visit: Flu-like symptoms\nDoctor: Dr. John Brown\nSymptoms: Fever, cough, sore throat, fatigue\nDiagnosis: Influenza\nPrescriptions: Tamiflu 75 mg, rest, and hydration\nNotes: Advised to stay home and rest, follow-up if symptoms worsen.\n\n\\# Allergies:Penicillin\n\n\\# Immunizations:\n- Influenza vaccine: October 10, 2022\n- Tetanus booster: June 5, 2021\n\n\\# Emergency Contact:\nName: Jane Doe\nRelationship: Wife\nPhone: (555) 987-6543\n###"
        },
        {
            "role": "user",
            "content": "Does John Doe have a flu vaccine and if yes, when was it taken?"
        },
        {
            "role": "assistant",
            "content": "Yes, John Doe has received an influenza vaccine. It was administered on October 10, 2022."
        },
        {
            "role": "user",
            "content": "How about a Covid vaccine?"
        },
        {
            "role": "assistant",
            "content": "The information provided does not specify whether or not John Doe has received a COVID-19 vaccination. Please check with the healthcare provider for updated records."
        },
        {
            "role": "user",
            "content": "If not mentioned, probably he didn't have it."
        }
    ]
}

Given the Request body above, the LLM will answer to the last User message. Below is the Response Body, which contains some system information like Request ID, Created Date, LLM Model and Tokens used, together with the Response message itself.

{
    "id": "chatcmpl-328",
    "object": "chat.completion",
    "created": 1719580369,
    "model": "openhermes2.5-mistral:latest",
    "system_fingerprint": "fp_ollama",
    "choices": [
        {
            "index": 0,
            "message": {
                "role": "assistant",
                "content": "That is correct. Since there is no information about receiving the COVID-19 vaccine in the patient record you provided, we can assume that John Doe has not received it, or at least it was not recorded here. Please check with his healthcare provider for the most up-to-date and accurate records."
            },
            "finish_reason": "stop"
        }
    ],
    "usage": {
        "prompt_tokens": 27,
        "completion_tokens": 64,
        "total_tokens": 91
    }
}

💡

All of the above is transparent, as APEX automatically extracts and displays only the Response message and keeps in the logs only the important system information like number of tokens used (useful for any billing and traceability purposes).

How the new APEX AI features work

Comments (2)

More from this blog

APEX - the state of things - April 2026

Oracle APEX Drawers for mobile apps

APEX Responsive CSS Classes

Running Claude Code with open-source models locally

Avatar image in Oracle APEX

Why would you care?

APEX in the AI era

APEX Generative AI Services configuration

Deep dive into AI Assistant component

Command Palette

Comments (2)

More from this blog

Why would you care?

APEX in the AI era

APEX Generative AI Services configuration

Deep dive into AI Assistant component