CRM Observations & Reporting

I have spoken about building a CRM for a client in the past. This CRM will be open sourced (pretty sure, topic for another post). It is still being used, and the feedback so far is that it's quite helpful. Today's post is an update on some of the features I delivered.

Observations are dreams

It seems like everyone is coming to this idea of having dreams (the adjacent possible) and I am no exception. In our CRM system, we capture this interaction data, and build out customers and relationship data as it is processed live. However nothing to date was capturing trends and feedback about system behaviour. I designed this concept of observations that the system runs on a cadence in the background. It is a separate agent setup with its own tools and instructions to capture bigger picture items from the interaction data.

Because of how we structure the agent context, the last X number (usually 10) of the user's messages are in the context. As a result, it is not imperative that bigger picture items and trends are picked up right away, since the agent can infer from the last few messages anything critical (feedback, preferences, etc.) - crude, but it works. This observation system is designed to be slower and more purposeful and so I am ok that it runs in the background once a day for everyone.

The analogy to dreams is clearly a stretch (if not overly ambitious) but the rough principle is the same, in that the system doing some re-jigging overnight while it is quiet to hopefully self-improve.

People still need their reports

One feature I had been touting to the team is scheduling data delivery. Say the sales lead wants to know how many interactions occurred across the team at the end of each week. Wouldn't it be nice if this just ended up in their inbox or app at their defined cadence? Delivering on this turned out to also address immediate report-run requests, since both require solving the same problem: how do you safely (avoiding raw SQL and unprivileged exfiltration) expose aggregated data, a.k.a reports, in our system?

The unglamorous solution is for now yaml files where params are defined and templated SQL is predefined in a yaml file and parsed with a python function. We then have tooling to ensure the agents strictly pass all required data, and of course use Pydantic context dependency injection to ensure the agent cannot override which user the report is being run on behalf of. The agent can search for reports and have their params explained, but cannot modify anything (including writing a new report) that is not strictly defined beforehand.

I have some ideas about how to improve this safely such that we do not need to rely on effectively hardcoding reports, but as of now reporting will remain an admin function exclusively. The reporting seems to work, and that was the milestone.