On the 8th of June 2026, our team at sixn.ai flew to Singapore all the way from Colombo to participate in NEXT Hackathon 2026. This year’s hackathon theme was Agentic AI applications. Our project was Bloom, a multi-agent fashion companion that helps you build your dream wardrobe.
I am happy to announce that our team has become one of five overall winners and also won the category Best Use of Stripe.
Read about it here: AI company sixn.ai wins at Asia’s largest AI hackathon — Sunday Times
]]>I and my team at Sunchain Technologies had the opportunity to attend SuperAI 2026 held in Marina Bay Sands, Singapore and participate in their NEXT hackathon. This year’s hackathon focused on agentic AI and this is the project that got us up the big stage as winners.
Currently when you want to find an outfit for an occasion, you have to have several tabs open, browse through several brands and sites and build your outfit in your mind. If you are an absent minded person like me you have to screen shot everything and do the comparison as well.
What our application ‘Bloom’ does is lets you do this with the help of agentic flows.
First, we wanted to create a single agent that helped user find what they want for an occasion and/or an aesthetic. Then we understood the process of selecting outfits was not linear and simple as we thought. We often look at what the store has, filter things out, pick few choices, fit on, ask our friends for criticism and if we don’t like what we chose, we go back and browse items again.
To emulate this process, we had to create multiple agents.
Ability to select an occasion and an aesthetic trend you want to follow.
Browse outfits from several different brands and pick.
Virtual try-on the assembled outfit by uploading a full body picture of yourself.
Save and like the outfits you like.
Get recommendations for your preferences.
Bloom is a full-stack application: a Next.js 16 / React 19 frontend, a FastAPI backend, PostgreSQL with pgvector for semantic search, and a Chrome extension for on-page styling.
We created multiple agents to assist user.
| Agent | Task |
|---|---|
| Stylist Agent | Evaluates outfits fetched by Researcher, provides explanations to users and decides whether to browse databases, online catalogues etc (a ReAct loop). |
| Style Critic | Determines whether or not to proceed with stylist agents choices and if the choices does not seem satisfactory, re-initiates the loop. |
| Sourcing Planner | Determines which brands, web pages and catalogues to search as well as which keywords to use to fetch all the relevant products. |
| Supervisor Agent | Supervisor agent is an agent that runs at the system levels and supervises the tasks and context. Dynamically routes between steps based on runtime conditions, what the catalog yielded, whether the critic approved, whether the user has a budget set. It also makes sure that Stylist and Style Critic are not stuck in an eternal loop going back and forth. |
| Checkout Agent | The agentic flow that helps you checkout multiple items from multiple brands and sites with a single cart using Stripe payment gateway. |
In addition to the agentic flows, there are other core functionalities and tool calls that support this process.
| Function | Purpose |
|---|---|
| Researcher | Tool that calls Exa API to get weather, fashion trends, dress codes etc. Works in tandem with Sourcing Planner. |
| Personalizer and Preference Engine | Executes to logic to add preference weights to each item that fits user description based on user’s past preferences and choices. |
| Budgeting Tool | Executes logic to evaluate every single article of clothing and outfits so user can have something from their budget range. |
After the agents assemble an outfit, the user can see themselves wearing it. Bloom uses Google Gemini 3.1 Flash Image for virtual try-on , a multimodal model that composites the user’s photo with the selected garments.
The challenge: Gemini was morphing user faces toward garment-model faces (catalog photos show different people wearing the clothes). I solved this with an identity-lock prompt, a multi-paragraph instruction repeated at the start, middle, and end of the prompt that forces the model to preserve the user’s face, skin tone, and body from their photo while discarding any human features from the garment images.
The preference calculation is a weighted process.
Likes have less weight than Saves. And both likes are saves are weighted again by the recency. A users tastes may change over the course. Save done within last 30 days had more weight than those done in last 60 or 90 days. A Like or save done before this year had no impact and were discarded.
After taking top weighted Saves and Likes, we analyze the attributes of the individual articles such as size, fabric brand and also attributes of the outfit such as color paltte.
In addition we also take frequently appearing key words in product descriptions as attributes too.
then when user makes a search for an outfit, if the product descriptions contain those key words and fit the most frequent attribute values, they are scored. Top 3 scored outfits are provided as recommendations.
Since this is a demo done for a hackathon it obviously lacked lot of important details. Here’s what must be changed if we plan to move with this project.
Bloom demonstrates that agentic systems can be both genuinely autonomous and reliably bounded. The key patterns:
In the previous article, we talked about how we read spreadsheets without compromising accuracy or performance. But data is not really impactful if it is not visualized so today we will be discussing how we got our document analyzer to generate a chart.
The first step was defining a tool interface the LLM could reliably invoke. A tool call is when an LLM recognizes that it needs to perform some action outside of its current capability to answer the user query and needs to call an outside API or service. An LLM can only generate text by itself so a tool call will greatly improve its capabilities. In our case, the LLM has to generate a tool call to call on a function that generates charts.
To define a tool call, we need to look at the function (the tool). There are different types of chart that we need to generate based on the nature of our data. This function should be able to handle the basic chart types needed to visualize data. Then it needs x data, y data and names for x and y axis.
Here’s the pseudo-code for the function that I designed. It requires helper functions for each chart type. It can visualize multiple y data as well, depending on the chart type
FUNCTION draw_chart(chart_type, x_data, y_series_data, y_series_names, axis_x, axis_y, title):
CREATE a new chart object of type chart_type
FOR each index i from 0 to length of y_series_data - 1 DO
series_data ← y_series_data[i]
series_name ← y_series_names[i]
ADD series_data to chart using x_data as X values
LABEL the series with series_name
END FOR
SET X-axis label to axis_x
SET Y-axis label to axis_y
SET chart title to title
DISPLAY the chart on screen
END FUNCTION
In addition to the helper functions generating each supported chart_type, there are two other helper functions. First one ensures that both x_data and elements of y_series_data (y_series_data is a list of lists) have the same length. The other helper function is to check whether x_data is safe to convert to numerical values, and if so convert.
Now that the tool is done, let’s design the tool call. I used JSON to design it as it is a format that the LLM can infer easily.
tool_func = [
{
"name": "draw_chart",
"description": (
"Generate a chart (bar, line, scatter, pie) given x-axis values, "
"multiple y-series, qualitative categories, and axis labels/title."
),
"parameters": {
"type": "object",
"properties": {
"chart_type": {
"type": "string",
"enum": ["bar", "line", "scatter", "pie"],
"description": "The type of chart to create."
},
"x_data": {
"type": "array",
"items": {"type": "string"},
"description": "List of values for the x-axis (qualitative or quantitative)."
},
"y_series_data": {
"type": "array",
"items": {"type": "array", "items": {"type": "number"}},
"description": "A list of lists, where each inner list is a numeric data series corresponding to x_data. Use a single inner list for one series."
},
"y_series_names": {
"type": "array",
"items": {"type": "string"},
"description": "List of names for each numeric series in y_series_data (e.g., ['Sales', 'Profit'])."
},
"axis_x": {
"type": "string",
"description": "Label for the x-axis."
},
"axis_y": {
"type": "string",
"description": "Label for the y-axis."
},
"title": {
"type": "string",
"description": "Title of the chart."
}
},
"required": [
"chart_type",
"x_data",
"y_series_data",
"y_series_names",
"axis_x",
"axis_y",
"title"
],
"additionalProperties": False
}
}
]
Here, the “name” property is the same as the name of our tool function and the description is a comprehensive description of what the tool call is for. Then the most important part, parameters come in.
Parameters are what we are passing to the function. The LLM should follow the format of the tool call and organize our data the way the function requires. The chart types parameter explicitly states what types of charts we are supporting. Then the x_data parameter is an array of Strings as x_data can be either quantitative or qualitative depending on the chart type. Our tool handles this by checking whether it is safe to convert x_data to numeric data by using the helper function we talked about earlier. Other than these, the LLM also generates x-axis and y-axis names, the chart title and a description before sending these out as a tool call.
The parent function (LLM orchestration function) determines whether the response is a tool call or a regular response and acts accordingly.
]]>In our last article, I talked about how I handled concurrency in the RAG system. Today, I want to talk about another challenge that came across when developing this Document Analyzer system: reading spreadsheets.
Azure Document Intelligence supports Excel formats and can be used to extract information from spreadsheets. It can detect tables when they are explicitly and well-defined. But the catch is, for unformatted sheets, it reads each cell from left to right and just outputs the values sequentially, not preserving the table structure or column-row mappings.
One way to mitigate this is to loop the data and manually map them using row and column counts. But this massive loop will take some time for a large spreadsheet with several sheets and that could lead to user dissatisfaction. So I wanted to implement a strategy that reads tables fast while preserving the structure.
I thought of how we read tables for typical data analysis tasks, using Pandas, into a dataframe. Can we not store a dataframe in a vector database? Luckily for us, Pandas has this function:
df_string = df.to_string()
This converts a dataframe to a string and stores it. The LLM can then infer its structure without the aid of a loop. But we need a more refined algorithm to read the spreadsheets. There are two types of sheets: those that have proper Excel tables and the others that just have data without any table formatting.
When reading data without table formatting, we have to identify which are data rows and which is the header row. To decide which of the top k rows is the header row, the following metrics need to be calculated:
number of non null cells / total number of cells in the rownumber of UPPERCASE cells / total number of cells in the rownumber of short-text cells / total number of cells in the rownumber of numeric cells / total number of cells in the rownumber of symbol-dominant cells / total number of cells in the rowThese metrics are decided using the heuristic knowledge that the header row will not have null cells, is more likely to contain uppercase text, and is less likely to have numeric, non-alphanumeric characters, or short text (more than 50 characters long).
After calculating these metrics, we compare them with the row below (which we assume to be data rows at the moment). If the rows below are date/numeric heavy, then the candidate row gets a boost; otherwise, it doesn’t. This allows us to capture the distributional shift between header and data rows. Afterward, we penalize less header-like traits and reward header-like traits and return a single number as a score. Using this score, we decide which one is the header row among the top k rows.
Then we use the following algorithm to convert the data into a dataframe.
In our read_sheets() function, we load each notebook as bytes and go through each of the sheets looking for tables.
filename, sheetname, tablename, and headers as metadata.For spreadsheets without table formatting, where we have to calculate and determine what the header is, we also store the index of the header row in addition to the typical metadata.
When answering a user query, the LLM can not only pluck out the answer easily from the dataframe converted to string, but it can also provide formulas and other references to the columns, tables, and sheets containing information using the metadata.
Here’s the flow chart for the read_sheets() function:
The goal here was to retrieve data as accurately as possible, with the context and metadata, and to reduce the performance overhead.
To conclude, spreadsheets require a different ingestion strategy than unstructured documents in a RAG system. Instead of reconstructing tables cell by cell, this approach preserves structure by converting tables into DataFrames and storing them as strings with rich metadata. Lightweight heuristics are used to reliably detect headers in loosely formatted sheets without expensive processing. Chunking tables into smaller, consistent units keeps retrieval efficient and scalable. Together, this design balances accuracy, performance, and practicality in production environments.
]]>In the last post, we discussed how the architecture for the Document Analyzer was designed. Now we have to think about how to actually ‘read’ our documents. This article is about the challenge of reading large documents without taking too long and driving the users insane.
When I first started this project, I used LLM Sherpa’s LayoutPDFReader. But then I realized that this is simply not enough. It cannot extract text from OCR documents, and it didn’t support all file types, just PDFs. That’s how I got to know about Azure Document Intelligence.
from azure.ai.documentintelligence.aio import DocumentIntelligenceClient
A rookie mistake that I made is not knowing there are two types of DI clients. There is a sync client and also an async client. What the sync client does is when you are reading a document, it blocks execution and waits for that operation to finish. This will hold you up. But when you use the async client, it only initiates the operation and returns control to the program, allowing other operations to run alongside it. It still waits for the remote operation without blocking the entire worker. This way, the UVICORN worker will not be held up when you are processing a long document. So we have imported the async client here.
Then we need to load our environment variables just like we did before. And now we move on to actual text extraction.
Here is the function that I use to extract text. The result of this is either markdown text or a plain string of text in our document.
async def _analyze_window_markdown(
client: DocumentIntelligenceClient,
*,
pdf_bytes: Optional[bytes],
pages_expr: str,
) -> str:
if not pdf_bytes:
raise RuntimeError(" no pdf_bytes were provided")
req = AnalyzeDocumentRequest(bytes_source=pdf_bytes)
poller = await client.begin_analyze_document(
"prebuilt-layout",
req,
pages=pages_expr,
output_content_format=DocumentContentFormat.MARKDOWN
)
result = await poller.result()
# result should have .content when MARKDOWN is requested
if hasattr(result, "content") and result.content:
return result.content
# fallback if it returns a plain string
if isinstance(result, str):
return result
return ""
When I first implemented this, I noticed that it took so long to process a PDF and I wanted to make it faster. This is where I learnt about concurrency properly and this is also where this project stood different from my academic projects.
The solution I came up with for this was to break down the PDF into page windows and concurrently process those windows.
I used a local semaphore inside each worker to limit how many windows that worker could process in parallel. It worked but only within a single worker. With multiple Uvicorn workers, each one had its own semaphore. That meant total concurrency was:
number of workers × local limit
So instead of protecting the API, I accidentally multiplied the load. Not ideal.
To enforce a true global limit, I created a Redis-based semaphore:
This centralizes the concurrency limit, exactly what I wanted.
But then I hit the next issue.
Because each Uvicorn worker is its own process, they all ran the “seed tokens into Redis” logic. The result?
GLOBAL_DI_REQ_LIMIT × number of workers
Again, not ideal.
I fixed this by adding a simple SETNX flag in Redis to ensure that only the first worker initializes the token list. The others skip it.
Even after that, I ran into one last problem: workers were acquiring tokens during startup. That meant:
The fix was simple and clean:
Here’s the pseudo code of this logic.
multiworker_mgr.pyCONSTANT global_throttle=10
CONSTANT throttle_key='di_tokens'
CONSTANT token='t'
ASYNC DEF seed_semaphore():
IF semaphore IS None:
app.state.redis.lpush(throttle_key,([token]*global_throttle))
ASYNC DEF acquire_token():
IF NOT timeout:
token=app.state.redis.blpop(throttle_key)
IF token NOT None:
RETURN token
RETURN None
ASYNC DEF release_token():
app.state.redis.rpush(throttle_key,token)
Here, in this code, we populate a Redis list with tokens according to the GLOBAL_DI_REQ_LIMIT.
The blpop operation we do when we get a token blocks the coroutine if the list is empty and once the list has an element the operation can proceed.
document_processor.pyASYNC DEF anayze_windows_concurrently():
ASYNC DEF worker():
AWAIT acquire_token():
IF token IS NOT None:
TRY:
md=analyze_window_markdown()
RETURN md
EXCEPT:
RETURN None
FINALLY:
AWAIT release_token()
sem=asyncio.semaphore(3)
ASYNC DEF limited_worker():
WITH sem:
RETURN AWAIT worker()
result=AWAIT asyncio.gather(limited_worker(w) FOR w IN windows_list)
//windows_list is a list with tuples (starting_page, end_page). its the page range of the window
RETURN [(window_page_range, md) FOR md IN results]
In analyze_windows_concurrently, we create an async worker that acquires a token, calls analyze_window_markdown and then releases the said token when analysis is done. We set the local semaphore to limit the no of windows the worker is allowed to concurrently process. Then inside the limited_worker, we call the workers with the semaphore.
asyncio.gather concurrently executes the limited_worker and gathers the result.
Not only this allowed me to concurrently process the documents, this logic can also be scaled by changing throttle limits. This is point where this system moved from just ‘working’ to ‘working reliably in production level environments’. And this is also where I understood the difference between an academic project that I did as a university student vs. industry projects.
]]>Retrieval Augmented Generation (RAG) is when we use data retrieved from an external data source (such as a vector database) to enhance the response of a Large Language Model (LLM). The main reasons to do this are, first to provide the model a better context, secondly, to increase the factual accuracy of the answer.
Where there is no retrieval mechanism, to answer a user query, the LLM solely relies on its pretrained knowledge. This knowledge cannot be considered completely factually correct. However, if we can fetch this knowledge and feed it to the LLM to enrich the response then we can get a better response.
Now, let us move on to document analysis. Confidential documents such as reports, contracts, spreadsheets etc. needs to be analyzed and cross referenced sometimes, rather than doing this manually, the aim is to leverage the power of AI and automate this task.
A major concern when dealing with confidential documents, is privacy. This is where sessions come into this design. Once a user session expires, all the data such as uploaded documents, raw text, vectors, chat history etc must be cleared. Not only that, one user should not be able to access another’s data under any circumstances.
Python FastAPI
Using Python for any AI/ML related application is self explanatory. So, let’s discuss why FastAPI specifically.
FastAPI is built on ASGI (Asynchronous Server Gateway Interface). What that means is it supports asynchronous and concurrent request handling. FastAPI also uses Pydantic for schema creation and type handling, give you control over what your data looks like and validating it. Personally, I also found FastAPI to easier to grasp, with little to no learning curve.
ReactJS
This one is also somewhat self explanatory. FastAPI integrates well with ReactJS, making API connection smoother. ReactJS has a component based architecture, which makes it easier to integrate new features into the UI and also provides efficient rendering.
Redis Session Database
FastAPI is created with concurrency and scalability in mind. And this is achieved by deploying several FastAPI workers that can respond to requests simultaneously and handle traffic. But each of these workers are a separate OS process with their own memory and cache. But to sync the state of the whole backend server, they need a shared memory. This is where Redis comes handy. Redis acts as the shared memory for the whole application.
Another reason is, in order to stop the workers from overwhelming external services like Azure Document Intelligence, we must pull a request cap. Inorder to make all the workers abide by this, we need to utilize Redis. (We will be seeing more of this in the next sections)
ChromaDB Persistent Database
ChromaDB is a lightweight and open source vector database. When run in server mode, it is accessible as a separate process through a separate port. The actual storage could be configured to be a cloud storage, an SQL server or a local storage. In this case, we will be using the local storage.
Azure Document Intelligence
This is what gives our Document Analyzer the capability to ‘read’ documents. Document analyzer supports a wide range of images and documents types including OCR documents.
There were several key challenges I encountered while designing, developing, and deploying this system:
Follow along with this series to see how each of these challenges was addressed.
]]>