Chatbot Endpoint

When you enter a question in the front end, a POST request to the /api/chat endpoint is sent. The body of the request must include the question from the user, in the following format:

    "question": "the question goes here"

The response from the application is an event stream, as defined in the Server-Sent Events (SSE) specification. The events that the server returns to the client have the following sequence:

  • data: [SESSION_ID] session-id-assigned-to-this-chat-session
  • data: [SOURCE] json-formatted-document (repeated for each relevant document source that was identified)
  • data: response chunk (repeated for each response chunk returned by the LLM)
  • data: [DONE]

The client can choose to ask a follow-up question by adding a session_id query string argument to the request URL.

The high-level logic for the chatbot endpoint is in the api_chat() function of the Flask application, in file api/

@app.route("/api/chat", methods=["POST"])
def api_chat():
    request_json = request.get_json()
    question = request_json.get("question")
    if question is None:
        return jsonify({"msg": "Missing question from request JSON"}), 400

    session_id = request.args.get("session_id", str(uuid4()))
    return Response(ask_question(question, session_id), mimetype="text/event-stream")

The ask_question() function in file api/ is a generator function that streams the events described above using Flask's response streaming feature, which is based on the yield keyword:

def ask_question(question, session_id):
    yield f"data: {SESSION_ID_TAG} {session_id}\n\n"
    # ...
    yield f"data: {DONE_TAG}\n\n"

Share this article