# From Systems Engineering to Agentic Systems Engineering Every hackathon I see a common theme - More RAG and even more Chatbots! It's good that people are trying to learn Agentic Engineering but no-one seems to be interested in working on Distributed Systems no-more. It's no longer cool. I propose a solution. What if instead of building chat-bots, you work on an abstraction one level up which you will allow you to catch a bigger fish. See chatbots suck at scale, it's very difficult to manage context, you waste time just to find the relevant session, then you have to scroll dozens of page just to find the needle in the haystack. All this leads to mental fatigue. ## A Session is a human construct! Sessions were created to solve for context rot, in which unrelated data in your context leads to hallucination. It did solve the problem but created another problem of session management. Normal humans can manage probably 3 to 5 sessions at a time and it does not scale well. So what's the solution? ## Listen to events! ![](https://cdn.hashnode.com/uploads/covers/64e9d89567deab13f465f320/68d72acc-73a6-494f-8e41-790e59efb8ea.png align="center") Similar to the Event Loop based architecture in Distributed System, we can listen to incoming triggers from configured sources, and forward it to a router. The router then decides what to do with the trigger. It internally has to find whether it's part of an existing thread (group of related events). If yes, pass it to the relevant worker which is managing that thread. If no, create a new thread and map it to a worker. The thing to note is that there will be sessions in workers, but you don't have to worry about it. Just like you don't have to worry about processes in your Operating System. You click an application, it opens and the entire lifecycle is managed by the OS. This is a similar abstraction but the mental model is completely different. ## How to manage context? With so many events coming in, a natural question would be "What about my precious context window?". How will the workers manage so much of the context? Compaction is not very useful as it can loose information. **Inline & Spill** is a superior approach which only maintains the required metadata in the context and then provide the ability to fetch the remaining information from a persistent storage/database on demand. This hybrid approach is very similar to RAG. To fill up the context, we could leverage the concept of time and space locality. - **Time locality** - What has happened recently from the rest of the env? - **Space locality** - Fill up the neighbouring events or augment the details of the thread. > Start small, add what is near and leave a pointer to the rest. ![](https://cdn.hashnode.com/uploads/covers/64e9d89567deab13f465f320/40e85e99-3217-4b2f-b14c-4fe461abc7ea.png align="center") ## Need for Knowledge Graph! To find the entire information on what and when has ever happened based on a trigger, we need something more than semantic search! Semantic search will simply not scale as it was meant to search only for similarility. In the above image, when someone posts "Hey the login screen does not work", the router has to decide whether to create a new thread or reuse an existing thread. If this is a new issue, a new thread is created, which would be assigned to a worker, which could take multiple actions to remediate the issue. In future, if someone asks, "Was the login issue fixed?", then we need to trace all linked actions associated by previous triggers. This leads to traversal of a graph like data structure and to build such knowledge graph is a core distributed systems problem. --- ## Segregation of Compute and Storage To achieve the above design, we would need an always-on router, sitting in Cloud, along with Highly Available Workers with their state maintained in a Durable Storage. This philosophy draws parallel with the classical distributed systems paradigm of segregating compute and storage so they can be independently scaled / managed. --- The goal of this article was to let you introspect on how could you build such always-on agent which listens to a variety of events and takes actions automatically in a trusted scope. This blog is intentionally ambiguous and does not provide implementation details. Hope you found it useful. ![](https://cdn.hashnode.com/uploads/covers/64e9d89567deab13f465f320/fb9e1b44-879f-4fc5-866e-405b823ac8ec.png align="center") ## Appendix - - -