agents-at-work-17.txt

gistfile1.txt

Speaker 1
00:00 - 05:49
Hi everybody, this is Jordi from Agents at Work and today I have the pleasure to be with Rodrigo. He's an economist, worked for five years into BC land. He was a programmer since he was a teenager so he decided that he was going to try into tech and it was 10 years since then that he has been like into like really deep tech. Hi Rodrigo. Hi, how are you doing? Doing well, thank you. So your story it's already interesting from where you started and how you ended up or now you're in the middle of the story but you know like where you are at now how was that yeah i i took a con in graduation right and then i thought i was going to go into research so from the beginning i always loved theory right and i tried to build my own theories in the first year but by the end of the second year i watched a presentation about venture capital from monash's capital which is a i think it's the foremost fund manager in Brazil for venture capital. And by 2011, they were beginning, right? So that all made sense to me because they were investing in technology, which I loved, and I always loved computers. But there was a business side to it. So it competed against the idea of research in my mind because it seemed like I could get into a business area, which was also intellectually interesting because most, I think things have changed in venture capital, but by then it was still a matter of having like a brilliant idea that you could implement. Right now, capital is much more important and so on, but by then it was still a romantic age of venture capital, so to say, for me at least. Yeah, 2010 until 2021 was that era. And I worked in consulting too, So I did a bunch of data analysis and number crunching, and I always use macros to improve my work. Sometimes I reach for unconventional analysis. There was this project for a bank that I did a triangulation of all their agencies. And I think you're kind of bank agencies, right? And if you have a triangle, you can remove the middle point and it's still covered that area. So I always like analysis by itself. And after a few years working in business, I went back to work at Monash's Capital again. And by then, the fund had grown so much, but we were still controlling our investments, our portfolio using Excel spreadsheets. So I reached out for Ruby on Rails and I built a small investment management system. And that's when I realized I love tech and I need to do more of it. And the natural next step was to try data science because I have the education in econometrics from economics. So I could enter in tech with data science and modeling, right? So I went to a portfolio company. And from that portfolio company on, I always worked in tech. During a few periods, I worked mostly with software and mostly prototypes and trying new stacks for customers and clients. And I also worked with a big machine learning team at Loft, a prop tech in Brazil, where I did a lot of real estate price modeling, other models as well. And I did formal methods because at the same time, I was pursuing a master's degree in computer science. And I started a small startup to compete with Carta in Brazil. So we did stock options management. So what we needed in my platform was to have the platform should resemble a cap table in its design. By the time that there was an industry group, the Open Cap Table Coalition, and they proposed a format for representing cap tables, which are how startups show who owns what in the company. Because when you're buying or signing companies, you need to know who are the owners to at least pay them correctly. So it's a matter of data integrity. And when you start exchanging cap tables, Excel spreadsheets, it all costs a lot of time in lawyers, for example. Yes. So if you have a standard to exchange cap tables, it should control that problem. But once I looked at the example data, I realized it was invalid. So I tried to look at how could I use their format to validate the data. I can, if that format works, I can use it at my company. And the format was written in JSON schema, which is only syntax. It has a format, but it has no business rules. But what we want in a cap table is, I mean, you need a series of events. Someone invested, someone sold a stake and so on. So it needs to be a transition system, a state machine. And you cannot express state machines in JSON schema. You can give the shape of the data, but not how it changes. So I used that as my topic in my master's thesis. And I started working on a format for cap tables that would be the same as a state machine. So I could verify it. And then I could check the properties we want of an accounting system, which is it respects accounting identities and it doesn't allow for double spending. Interestingly, the damn design is very similar to blockchain, to the UTech sole model. And I use that as a basis for my system at my startup. So that's where I studied formal methods. And you see like all these problems that seem really simple, like, well, you have an Excel for this, like what's the problem? You start modeling it and you're like, holy, like this is actually hard. Yeah, you need to pay very well-paid financial analysts to do so. It's not cheap workforce to handle spreadsheets. So it's not simple, it's not trivial. Things are not tacit when you do spreadsheet modeling that you need someone to know. And if you don't tell them, they won't know. And then you mentioned formal methods. For me, something works because I have some tests and I run them. I say, you know, this is Python. This unit test works. So that means that the program is correct. What are formal methods? Because I think that they are not tests in that way. 

Speaker 2
05:49 - 05:50
They are like the next. 

Speaker 1
05:50 - 12:12
They are not tests. Formal methods is when you prove things about your program and it doesn't matter what the input is. Basically, let's say, when you do testing, you try to give certain examples of the function trying to test. And let's think of the problem of triangles. You need to tell if a triangle is isosceles or a rectangle and so on. And if you have a right rectangle and you extend one side, it will still be a rectangle, right? A rectangle triangle. So there's some range of values for the side. They can change, but your output will be the same. So basically you need to find the inputs you want to test so to get all the outputs on their side. So you need to test for an equilateral triangle, an isosceles triangle, and a rectangular triangle. So you can divide your test space in boxes you need to fill. So when you do so, you are covering all your cases, right? And when we do this, it's not the same as covering lines of code. We are actually trying to cover each and every decision your program makes. And that's a principle. This is an exhaustive analysis, right? You can, if your input domain is finite, you can search all over it. But typically, the search space is still huge. So what you try to do is to think of what is correct about your program. So let me say a simple one. When you concatenate two lists of size N, M, M, N, of course, your output must be N plus M. So if your whole program works on such a list and this property is still valid at the end of it, it's very hard to get such a specific property right and the rest of our program wrong. So information is like your program is a variable in an equation, And the equation tells us about product properties. And you try to solve for your program for the property. But on a more ground level, it's just like doing logic on a program. So every time if X comes this way, this will happen and we rule out these cases. But you do so systematically. And that's pretty different of how most of the engineers relate with code nowadays. Where are formal methods used? Yeah. In engineering, I think we can give you two examples. So there's one example that doesn't look like formal methods, but it's pretty close in spirit. And there's a second example, which is formal method in the industry. So I think most engineers can relate to operator research. So maybe you're in a factory and you need some part to visit certain machines under some constraints, and you can build a problem when you need to minimize the cost of something, right? So I can use the software I use for formal methods to model such problems, because it's a problem of constraints, right? But we have constraints about numbers, not about programs. So when I worked in PropTech, we had thousands of customers going to visit thousands of apartments and we needed to create itineraries for the brokers to take them. And up to a point, we were doing so manually, but then it started not to scale. So that's the famous problem, the traveling salesman problem, right? Which is how can I visit these 10 cities and there's such and such roads for them in the shortest amount of time? I think a few engineers, maybe in robotics, might have come across this problem. And this is the first problem that we can map to formal methods very intuitively, which is you give a specification of your problem and constraints, and you have just a piece of software that solves it. So the same way in formal methods, like when you're analyzing, you do the same. You give your program in a language, and generally you need to use a specific language that a formal methods analyzer or solver can understand, and you specify constraints. So you might write the concatenatorless function and you can write down the properties and then the server can check if your code will always obey the property. It's like the server is proving it for you. So you don't need to write the proof by itself. And again, to bring that to industry practice, most big tech companies will have a formal message team because they need to have complete trust over their stack. And sometimes they will need to customize a few components in their stack. Let's say Amazon needs to optimize their GZIP library because on Amazon scale, it amounts to a few millions, right? So they might try to optimize it, but then it might make it unsafe because it's very low-level code. So they write and verify bits of their low-level networking libraries, and there's a team for that, right? Yes. I have seen it used a lot on hardware verification. hardware seems to be one of the main users of this then everything that has to be extremely safe so systems that are really complex for example in an airplane and then you just want to make sure that for example like if two things never can be activated at the same time but they depend on thousands of variables and you know like it's some kind of complex function you cannot just say like well yeah i think like i run some tests that's not tests are not enough yeah and you get you can map that to agents too, because when you speak of formal methods, we try to prove two things ultimately, which is safety and progress. So when you prove that the program is right, you prove that it always progresses towards its goal and it doesn't make mistakes. So it's always about safety and progress. And you need that for AI agents. You need to know that we always do something so there's no way the agent will not know what to do. That's a safety property. And you might pose the goal of the age in such a way, like a goal and sub-goals, and it won't repeat something that it all, some sub-goal that was already achieved. So that's progress too. You always pick a new goal. So you can start to mix concepts between different fields. And I think that's something that I got from economics because you do a lot of agent modeling economics. Economics is very much about proving things. I saw no numbers in economy. I saw only equations. And you get game theory which deals with agent behavior. So I started mixing the ideas, right? That makes sense. And actually, that's how I met you because it was on Reddit and I saw someone using Prolog. So all this stuff sounds like a lot of like heavy machinery that AI agents could use for mapping, doing the spec and then using it. For me, it's crazy because you can see that right now, like this is an afterthought. So for many, many years, I was actually this kind of machinery. You have formal methods, AI, what is that? 

Speaker 2
12:13 - 12:17
It's about like having some ontology and then, you know, like some machinery that crunches 

Speaker 1
12:17 - 13:12
and says like, okay, yes, this is safe. This is not safe. Tries to map stuff. But we started with this probabilistic of reformers learning or deep learning in general. And then the people were not overlapping. They were like, well, this is amazing. I just threw data in and it's able to extract some kind of probabilities. and then it's able to map things that seem to work well enough. And actually they do. But many times you can see how they have problems to do really simple, repetitive tasks. You can see this a lot when the strawberry problem, when you say like, okay, strawberry, how many R's are there? Yes. And they are not good at saying the right number. No, they're good at saying something that won't surprise you because they are optimized to lower the perplexity score, which is how much it should surprise you by the answer. So it's just plausible, right? 

Speaker 2
13:12 - 13:13
It looks like the answer. 

Speaker 1
13:14 - 13:48
But the probability distribution here, it's really hard to capture. And the crazy thing is that for these kinds of problems, you don't need a probability distribution. Like you can give me any kind of random string, I can count R's. Like, wow, how is this possible? You never saw anything similar. I've never seen a string with 1 million characters. How was I able to do it? I said, well, it doesn't matter because it's not a probability distribution. It's like I apply the same thing and I'm able to go character by character adding R's. So I don't need a probability distribution from there. And we have been developing this kind of thing since the 50s. Like what you say about constraint programming, this was a big, big deal. 

Speaker 2
13:49 - 13:55
Then with Prolog, this idea of, well, we have an engine that has some kind of constraint or premises. 

Speaker 1
13:55 - 16:46
And it's able to say like what's true and what's not. And I don't see enough people mixing both of them. So I got really happy when I saw like, well, there's some people doing it at least. How do you started mixing these two concepts? Yes, I was working with inbound agents, like sales agents, and it was like a Greenfield project. So our first project was very simple. We have this agent that represents a company, cars dealership, right? And it has an inventory of cars. And we want people to see the car in our website and paste the link and it should give more information and try to book a test drive for the customer. So the first choice was to use the file search from OpenAI. So we would just upload a PDF file with the inventory, but it didn't work. It really came up with hallucinated cars, which is because I think OpenAI just chunks the document and then moves it to a vectorial database. But I don't know how the recovery is made. It's not like it's cars indexed and a reg fragment that you can search and get the index back. So it just hallucinated. So my next idea was to use tools, right? So a tool talks to the database. That's perfect. So it works for some time, but pretty soon you need to start writing better tools because you don't want to move a generated SQL to your database by any means, right? So you start adding new tools and then your clients start using the software and they start complaining because sometimes it still hallucinates. That's because your tool was not well prepared to handle for diacritics, for example. So you get a bug, right? But those are the easy kinds of bugs. You can see the car name with a diacritic and it's not with a diacritic database and you fix it. But then more problems start showing that you can't control on the simple measures because the agent is not being faithful to the instructions on one hand and it's not being faithful to facts on your head because it's hallucinating. And how do you fix that? You can use better prompting, right? But it will always mix things up. It will always. And you start adding observation tools so you can see what tools were called under which context. And you get puzzled because the context there, everything's there, but the tool power is wrong. So from my difficulties in trying to get to a high reliability level on inbound agents, we started trying to like use a state machine to coordinate it. And then I wrote a prompt, which was a prologue prompt, the first write. So if I can write my prompt as a set of prologue rules, Maybe the LLM will make sense of it in a way that it's ambiguous when you start writing prompts. And you need to consider you're writing prompts for tens of customers. And the prompts need to be consistent between themselves. When you say a prologue prompt, do you mean that you wrote like a prologue program? 

Speaker 2
16:46 - 16:47
In the prompt, yes. 

Speaker 1
16:47 - 16:51
And that's the thing that you fit the AI? You said like, well, here is... 

Speaker 2
16:52 - 16:55
Yes, I use this in a different mode back in the master's thesis 

Speaker 1
16:55 - 16:59
because I started communicating with ChatGPT using formal language. 

Speaker 2
17:00 - 17:02
So I would write to ChatGPT as models 

Speaker 1
17:03 - 18:07
because a formal model, it's packed so much in a small description. I thought it was the most efficient way for me to communicate with the machine. So whenever I would try to specify a feature to VibeCode, I would give it a Dalloy specification. Dalloy is a form of methods language and you can model relations and tables and state transitions. So I give my specifications to VibeCoding in formal methods. And it seems to work because it packs all there is to know about the program, the specification, the simple syntax, right? And if something's not there, that's why the LM would do different in the code. So I do the same with Prolog. Maybe if I speak with Prolog to the machine, it would understand me better. And you can start very easily. You can just give your agent prompt and ask GPT to translate it to Prolog and see how it looks. And the way Prolog, I don't know how many of the listeners are familiar with it, But Prologue is very simple. You have rules in order to, a rule has goals and sub-goals. So if you need to achieve a goal, you need to achieve sub-goals and so on. So you might see it as in order to place a purchase, I need to know what item is selected and if the customer can afford it. 

Speaker 2
18:07 - 18:09
So it handles this kind of rules very well, 

Speaker 1
18:09 - 18:47
which maps so well to business rules because business rules are mostly small arithmetic calculations and if-else rules, right? So Prologue is a nice fit for it. So I After I tried running a few prompts with Prolog, it really amazed me how well it's worked. Much better than my natural language prompts. And then I started trying, well, what else can I do with Prolog? Can I make the LLM write queries? Can I make an LLM assert facts, add new facts to my knowledge base? And I think knowledge base is an important term here. But here, the important thing is that the LLM is not actually executing the Prolog program, or it is? 

Speaker 2
18:47 - 18:49
Like, does it have access to it? 

Speaker 1
18:49 - 18:51
No, I run a Prolog instance. from the description. 

Speaker 2
18:52 - 18:53
Yeah, yeah. This is extremely crazy. 

Speaker 1
18:53 - 19:02
Like the fact that with some probability distribution, it's able to act as, you know, like, well, yeah, like the computation would look like this instead of actually executing it, that I think that 

Speaker 2
19:03 - 19:05
it would be a safe way. Very different, right? 

Speaker 1
19:05 - 20:45
So Prolog could translate user natural language sentences to queries. I tested it working well. It can add new facts to the knowledge base and you can use a small library to run Prolog inside Python. Yes. That's where I, instead of like trying the components of it, like test unity, testing the components of the solution, I integrated it. So the end of it looks like this. Your agent receives a user message, the user input, right? And your agent knows your Prologue program. And your Prologue program is like a knowledge base. It has facts and rules. So knowledge base is a kind of hot term right now. Prologue is just a knowledge base in simple logics. What your agent can do is it reads in the user input and then it can extract facts from it. Let's say the user is saying they want to buy a car. So you extract the fact, intent is buy. And then the user might say they want a BMW and you might have a rule, a fact that states preferences, brand, BMW, and you start building like a database of facts. And then you can have rules like if the user has declared a preference and their budget and so on, you can then present the option to the user. So, okay, you write the program prologue, and then the LLM looks at the user message, looks at the program, and adds facts to the knowledge base. So by adding facts, we increase its knowledge, and what you want the LLM to do is to take an action, either to talk to the user or use a tool. So the action the LLM will take, which corresponds to use, is a query. You query it for the next action to take, and it recommends the next action based on the facts. 

Speaker 2
20:45 - 20:46
because they are logic actions. 

Speaker 1
20:47 - 23:15
So if the budget is larger than the price of the car, the user likes the car, the car is available for the drive, can book it, right? But yeah, the new fact might be stating that the budget is 50 grand and the car is 40. So it unlocks the next action because now you can prove it. The LLM adds facts, queries the prologue engine for the next action and then prologue runs. And you try to prove which actions the agent can take right now. And then the LLM looks at the action and can map the action to one of its tools because the action, the query might return fetch inventory, the query might return ask budget and so on as actions. So then the LLM sees it, it might either map it to a tool call and do it and then after the tool call, it asserts new facts or it might just return text to the user and the user will answer and you get it to loop again. So the LLM does the hard part, The very hard part, which is translating between natural language and logic. But the logic is deterministic here. It's all in prologue. And we were speaking of the AI winter, right? Yes. There's an immense amount of literature on ages and planets. I mean, I use this book. It's a book from Warsaw and Norway. And it's like a textbook of AI. But it's like thousands of pages of AI and knowledge representation and planning and inference. But when you look at the examples in the book, they are very much play examples because they are tricky to write. It's tricky to formalize knowledge. And there's this part because you need to translate the user input, which is natural language, to an structured form and then get it back from the structured form to language. And the modern AI can help us in two ways because it makes it easier to write a Prolog program because it can help you do so. So it lowers the barrier to entry to doing it. And I can do the very hard part of translating it in natural language and formal language from and back. And I started doing some experiments and it blew my mind up because I saw how much work I needed to do to get an agent working well, a multi-agent working well. And I don't even need multi-agents in a prologue program because I think when people do multi-agents, they are essentially doing a state machine. In this state, I have these tools and prompt and so on. And you don't need that in prologue because you have full control of the state machine. So I think that it has full control and also with some machinery that it's bulletproofed. 

Speaker 2
23:16 - 23:19
Like we know that it's doing the right transition as it was coded. 

Speaker 1
23:20 - 23:49
Yes, we can analyze the program because it's a logic program. You can prove it. You can prove inside Prologue that it will always return one action. And if you make that action and be talked to a human, it always do something safe. The axon of the program is like, if you can do anything else, it's called a human, right? So as you mentioned with formal methods, linear programming, constraint programming, logic programming, we developed like some crazy good engines, like symbolic analysis. 

Speaker 2
23:50 - 23:52
Like we did a lot of stuff. 

Speaker 1
23:52 - 24:01
Say like, wow, we have these machines that are able to start with some things that they consider true. Then they have rules to keep doing the transitions. 

Speaker 2
24:02 - 24:07
and then if you ask a question, for example, is this true or can this happen? 

Speaker 1
24:07 - 24:11
They are really good at saying like, well, let's start with these things as if they were axioms 

Speaker 2
24:11 - 24:19
and these rules and let's start deriving and exploring what kind of things we can arrive. And then they are able to like, there are like some limitations, no? 

Speaker 1
24:20 - 25:33
On computer, like what's computable. But there's many things that they are able to cut the tree in a lot. It's like, oh, I am really good at being close. They are also really good at saying like, actually, I explored enough to tell you we cannot reach there because we were able to reach there before we should have happened to in these three middle steps and I was not able to get one of them. So for sure you cannot get there. So there is a crazy machinery there. And the real problem was that, well, for the machinery to start computing, you need to give them the rules and the facts. Yes. And then people were the bottleneck. They were like, how do I do this? And you can see this a lot on constraint programming. Many times it's really easy to kind of formalize in normal language what you want. So if you are a factory and you have like three ovens and you want them at maximum capacity, you're really good at saying like, well, an oven, it's occupied. If it's doing this, it takes like this amount of minutes for baking different stuff. But then you say like, and I want to know how should I bake it? And they're like, wow, this is hard because there are so many ways to do it. And then logicians were actually like in a really good position because they were the gatekeepers. They were like, oh, do you know who knows how to make this translation? Me. And there were like really little amount of people that were able to do this. 

Speaker 2
25:33 - 25:44
But nowadays, AI makes every one of us able to just stay like, well, let me tell you the rule in the oven. And they're able to put this in the syntax that Prologs needs, plus make sure that the 

Speaker 1
25:45 - 28:29
facts are there and the rules are there in the way that Prologs understands. And then to just run Prologs. As LLMs, they have been trained on so much text. They know the formal languages. So it can help you with Prolog and other formal languages. And there are ways to get into formal methods. Like you might start with Prolog. And there's a power of Prolog website, which is amazing. I should point to that. But there's also a side of formal methods, which is focused on modeling concepts, which you have like the ontologies and thematic web. But you also have system design language, which you spoke of hardware design. So you can verify hardware layouts. but there are some languages that allow you to verify software designs. So there's two of them that I think that are more approachable. There's one called Alloy, like an alloy of methods, but alloy, which is, it looks like you're writing like classes, like with fields and methods, but then you put constraints on the relationships between objects. And if you download Alloy and look at a few examples, you see that you will be able to play with some problems that I play with. So he said something that reminded me, Writing down your problem is the hard part. You can specify the behavior. You want something to happen, writing it down, even for a recipe, is hard. And alloy lets you write down your system and then you can state some property and try to prove it. And you can work on little problems and still see even little problems are interesting. Like the problem when you need to cross the river with a cow and a lion and a tiger and one cannot eat the other one. And you can model that in alloy and see it work and explore the solutions. And you said something about transitions and exploring the states. So it's a good approximation to formal methods as well, because all the formal methods you can see as ways to do exhaustive analysis more efficiently. So you could check all the space, all the possible states of the program, and all the transitions, and see it will never reach such state. But you write more specific language, so you can prune the search space, right? Yes. Something that is interesting here is this idea of until now, there was like really little amount of people that were able to use these things. And many times, even they use some patterns. So if you take someone who is smart but has never seen formal methods before, they usually fall or they have never codified or encoded a problem in constraint programming or something like that. They fall always in the same problem. So, for example, they see three constraints that they need to put there. And then someone who knows, it's like they are forgetting one that it's key. And they know that because they have done it many times. And then you see the solution. It's like, you know, like you specified that you should select never more than one, but you didn't put at least one. 

Speaker 2
28:29 - 28:33
So, you know, when you run it, the former method returns something that it's like, 

Speaker 1
28:33 - 28:37
okay, yes, like there's a solution for this. All of them are off for saying something. 

Speaker 2
28:37 - 28:38
And you're like, all of them, they need one on. 

Speaker 1
28:39 - 29:50
That's something very cool about Alloy because Alloy is very visual. So you write a little model on the left pane. And what Ally does is you give it a schema with constraints and you try to find valid instances of your schema. So it's like you give the schema of your tables and you find valid data for a table, given constraints that go beyond what SQL can do, right? And then it can sample instances of a problem and you can see the instances. And then you see, well, it's impossible to have this loop between those two entities. And you can explore the solutions visually and that gives a very good intuition because you might think your model is good, but when you go through things such as it generates, you see, well, this doesn't make sense. And then you refine the model, right? Yes. So that's a thing which is different in alloy from other formal methods. It's a lightweight formal method, so it's a good entry point. And what you said, it's also interesting that it's this idea of LLMs are really good at this, out of the box. Like if you take ChatGPT5, do you know any people who are working on agents like fully like leveraging this power? So for example, it's like, well, I don't care if 10GPT5 is good at this. How good can I make a small model? 

Speaker 2
29:51 - 29:53
Yes, I was thinking about this today. 

Speaker 1
29:53 - 31:01
I've been reading in terms of some true documentation, some papers in what people are calling neurosymbolic AI. Basically what I'm seeing is like trying to extract facts from, but you just have about something more specific, which is? Like the LLM actually being good at encoding. saying like we have LLM trained for recording. I would like to train it to translate specific user inputs to product terms, like fine tune it for product. I think it's very reasonable to do so. But since it's used on so many domains, maybe when you reduce the model, it loses too much semantic about what exists in the world. Because when you write it, you ask it to write a program about how to do car sales. It relies on what's in the LLM, right? So if it gets small, maybe it will map, not so well. But I think it's a target for fine-tuning, but I haven't seen anyone do it specifically. Let's fine-tune a prologue driver. And what you said about this new research field of neurosymbolic, what kind of things are they doing there? Like, for example, you said that they extract facts. 

Speaker 2
31:01 - 31:03
Where do they store those facts? 

Speaker 1
31:03 - 31:36
Because, well, the reality is that there is infinite amount of facts, but I only want the important ones or the relevant ones. as the constraints that are relevant for my problem. Yes, that's what I see where it gets difficult. Because if you just run naively, like give text and ask it to extract facts, it would extract things that are really obvious. So what I see people doing is you have like AI input and then you give it some schema. So if you give a schema beforehand, it knows what facts to look for. And there's work on building ontologies and not trying to extract facts, 

Speaker 2
31:36 - 31:38
but to extract ontologies from the facts. 

Speaker 1
31:39 - 32:11
then you can feel in facts that are more meaningful. And that's what I see people are trying to do. And it's basically trying to map some kind of neural network to produce sentences in formal logic and run into some other component, like try to predict the formulas and so on. I think it's hard to try to use the AI to do the NLU and work with logic on the other side, but it's not necessarily restricted to agents or prologue, for instance. Yeah, when I say prologue, it can be anything. We can do also like colonize with Z3. Or all, and you can. 

Speaker 2
32:11 - 32:13
Because for me, for a long time, 

Speaker 1
32:14 - 36:28
people did knowledge extraction from text, but it was so brittle because you need to parse the text perfectly. And with LLMs to do the parsing, people are looking again at these problems. Like, it's the same technology, all RDF. Yes, even in our case, like my company does a lot of things on the agentic commerce realm. And we use LLMs in many, many parts of the system. And one that was actually extremely good was the idea of, well, the user, I don't want it to put the filters by clicking or by putting it in some fields. Like we are going to extract the filters from natural language. And we are the best ones to build the filters because after all, who knows the system better than we do? So it happens that they just tell us like natural language and we're able to extract the right filters from there and then know the right query. Yes. So I guess that the same is happening a lot of times with cleaning data. Saying like, okay, I want to extract, like I want to scrape some stores. How should I organize this data? And how should I store it? Yes. Have you seen more people working in that direction? That always happens. Always happens. Because since you will never be able to send raw SQL to a database, you always need to think about filters. I don't know if there's a general solution to it. But in any agent you try to develop, it will show up sometimes. So you might start thinking about state. So how do I record that the user has such preference? The ways I've seen it done is via two calls relying on a prompt or by striking instruction outputs from the agent text and storing the filters. But the question is how you write your filter language. In the end, you write a small domain language for running queries. So there's some art to it. You might write something like filters. I have fields, possible filters, but then you start adding disjunction and conjunction. So I can look for a cheap BMW or for expensive Vox. And it gets messy because then you have to output recursive data structures from your LLM query. And Prolog can do that so easily, so much nicely. Knowledge base can do it because you can change those kinds of filters in a way that you don't do so on SQL. They became logical queries. The ones that they see that are trying to get into this trend, more than the people from Formal Logic, because they are like basically like academics. Like there's not that much industry, but there are like some other like with knowledge bases and knowledge graphs and knowledge graph database. They are always trying to capture this stuff. And then for many years, they were able to say like, we can do so much stuff, please. Build on us because we can improve your product. But engineers saw it and said, this is so different to what I am used. I don't want to lose like months for doing an integration, but now LLMs can do it like pretty, pretty quickly. And I think that people are really sleeping on this. Yes, they are. Graph databases, if you look at them, they are knowledge database because there's something that unifies them. When you have a predicate in logic, like Rodrigo is the son of Mario. You can see Rodrigo is a vertex, Mario is another node or vertex, and being a son of is an edge. So if you have a knowledge base in logic, you can map it to a graph database. So it gives you a way to implement logic back-end via graph databases, right? And the opposite is true. If you have a data, if you are accustomed to work with graph databases, then you can work with logic because it's the same. Of course, you have a relation, a predicate that has three arguments. You need to encode it into graphic two edges and nodes, but that's pretty much it. So I think while both are being taught, speaking about at the same time, people are talking about both right now because they're the same thing. Yes. And it's important that there are many good ideas into this. Like if you apply the same thing to a lot of stuff, Google was really famous for just taking page rank. It's like, listen, here, this thing of Mark of Chains, like we can do like really crazy things. But you can see the same with what you said about knowledge database. Like people see that knowledge, no? And then it's like, you are able to map a relation. But then in 2018 or so, Google open source Zanzibar. And it's like, well, we have a new way for doing authorization. And you have nodes, and then you have relations. 

Speaker 2
36:28 - 36:31
And then if you are able to go from one to another, you're like, this is... 

Speaker 1
36:31 - 38:30
Yes, this is very cool because... Good. One application of Prolog in the industry is for authentication systems. It's very common. So, for example, you have Casbah, which is like a DSL for writing authentication systems. and a system for running it. Yes. You're writing the rules. So if this user did reset his password the last three days, so it's not old, and the user belongs to this group, then you can give it access to this resource to do such actions. So yeah, logic programming comes about very often when you see little configuration languages for authentication. But most of the people see it as separate things. It's good for knowledge base and it's good for authentication and authorization to be more precise. And it's like, no, no, Like, it's because they are the same thing. And I am sure that there is, like, a lot of stuff that we're missing or, you know, like, detecting things or images and things like that that are just applying these ideas. You might be missing it because I think software, I think, over the last 15 years meant, like, building applications, right? So what was hot in the 2000s? Well, Java was hot. Ruby was hot. People wanted to do web apps. Now web apps are kind of a commodity, right? And people want to do different things. It's natural that they need to learn different things. What everybody's looking right now is how to automate agents. So I think it's only natural to need to recall that body of research from the 80s and 90s. I mean, even machine learning, if you look at the models, like the neural networks, you can get 80s book on them. So the theory exists. Yeah. But of course, like there's a lot of engineering and then the scale that it's also necessary. So you said that most of the people doing this stuff are usually in academia or in really big teams, you know, like the Amazon team, the Google team, like they're pretty famous for the ones that do this. Everyone knows each other inside the industry. But is there any company that is doing this as a service? Who is going to be like I can see like many new use cases for this. 

Speaker 2
38:30 - 38:39
And what they have is the lack of knowledge and to be able to professionalize and put this to production. Is there any company that exists that usually reach out? There are quite a few. 

Speaker 1
38:39 - 41:25
there are quite a few interesting companies in the space. There are consultants, they do this kind of work. But for example, there's a small testing library. It's called QuickCheck, which is very famous. So it was written by a professor and he built a company out of it, which is KUVIC, if I'm not mistaken. And they test software for Volvo, right? So they have quite a good team of formal methods there to test if your airbag system will open. So there's one. There's also some companies that, They work in the programming language space and have quite a lot of work-independent types. And they tend to have that kind of work, which is like Twig or Siroquel or Galois, which is a French company. I think maybe in the aerospace sector and health sector, you can find companies that are specialized in very kind of software. Railroad systems work that way. It's just very different from what you get as software developer, because you would get into that kind of problem if you're an engineer that works in the rail space. That's what I meant. I think that there is a space for a new company that says, guys, we are not doing it for Roll Roll and Boing. We are doing it for everyone as a service. We are really good at this. What are you up to these days? Because before you said that you had startup, what are you working on today? What I really like about working with Prolog is that the syntax will map so nicely to a GUI because you can add a rule, add subrules, and you could have an editor for Prolog that people could use to build an agent. So I'm working on getting a small library to run agents on Prolog knowledge basis and try to link it to something that resembles a production server so I can start testing it to see how it works in practice. and I would like it to be something that a user can edit. So it would be a, well, the prologue looks like no code. Because if it's over, there's the trouble for you. Your programs will be very small and it's something you can edit on a GUI. And I would really see how it would work if you can design an agent just by building the rules and see it work. Something like, I'm pursuing it like as an open source project and doing my small research. But if anyone was to reach out and collaborate, I would be more than happy as well. So we make sure that, you know, like I tell the right people if I admit that. I think that's based for like a small library that can compete with PyDenticNA, you know, LangRef and all other libraries, but really works on a logic-based knowledge base, like for doing this kind of thing. Nice. Do you have already a name for it? Like people can find it in GitHub or it's under development yet? It's under development. I have a small repo with proof of concepts. But if you reach out for KB-Agents, there's my code and you can play with it. And you can play with Prolog and Python with it. Nice. I will make sure that I add the links there. So, you know, like the people who are interested in this, 

Speaker 2
41:25 - 41:28
like they can learn more about it. And do it outbound. 

Speaker 1
41:29 - 42:17
What kind of things would you like to see more in the next six months that you don't see happening enough today? Yeah. I would like to see more people playing with different components to LLMs and trying to break them down into something that's not magical because we need to start as an industry to provide value or people will start questioning the value of AI, right? So right now I see many projects which are just wrappers over OpenAI, but from all we've chatted about, you need more components and layers and logic to get an agent going and working well. So I think if people start to avoid those naive approaches, the industry as a whole will provide more value. And I've seen companies fail because they can't get an agent to work, right? And that's because the tools we have are still very much based on prompts. 

Speaker 2
42:18 - 42:19
So I like to do that. 

Speaker 1
42:20 - 42:32
That's important because most of the people say like, well, there's nothing wrong with being a rapper. And it's like, that's true as long as it works. And the reality is that many people are doing over-promising things that they are not able to deliver when the foundation is there. 

Speaker 2
42:33 - 42:37
It just needs the work to actually like, well, you need to go in the right direction and you need to work hard. 

Speaker 1
42:37 - 43:19
It's interesting because people can see there's a problem and they try to give a solution, which is a prompt. So a prompt is a black box. So try to cut a black box into two components and put a contract between them. It's so hard with prompts. If you manage to split a prompt in two and get it to work together on two calls, you essentially are doing engineering, right? So most people don't realize how hard is it to break a prompt into small manageable problems. And that's where AI solutions start to go wrong. Yes. And mostly also because LLMs are this like a probability distribution. When they update the model, you don't know if it works. 

Speaker 2
43:19 - 43:21
Yeah. I have that problem with updates. 

Speaker 1
43:22 - 43:58
So every model update is a risk. You need to have like a golden set to test. It can change your behavior. So that's bad, right? Because the GPT-4 to 5 was very difficult for prompting. But if you had a solution that was more broken down into components, more of that components would be reliable to a model change. Yes. And right now, there is no easy way to do it. Like, you don't know what's making it reliable or not. Yeah. What are the main weights that you say, like, well, the other ones can change. But as long as, you know, like this constraint, like it's kept every update, like we are good here. So there is a lot of vibe engineering. 

Speaker 2
43:59 - 44:01
Yes. It's dangerous. 

Speaker 1
44:01 - 44:02
Like engineering is dangerous. 

Speaker 2
44:02 - 44:06
You need to have a very good understanding of what you're doing with VibeCoding. 

Speaker 1
44:07 - 44:36
Because I'm yet to see a big project that used VibeCoding and didn't incur in a lot of refactoring or rewrites. And the flows I found productive to use for agent decoding, they are very costly in terms of tokens because you need to make it run your Docker Compose, to have an environment and look at the test data and that print statement. And it just turned out to be a lot. Yeah. And what would you like to stop seeing in the next six months? You know, if you can just like... 

Speaker 2
44:36 - 44:36
AI is locked. 

Speaker 1
44:36 - 45:43
AI is locked. YouTube is filled with auto-generated documentaries. LinkedIn is filled with carousels of pictures made by AI. And I like to see less of that. Less people using AI to write or AI to post. Because it, frankly, you can tell when a text is bland. AI text is bland. And I am worried because I think that people are being trained on this. So it's not only like the machine. Like people right now, like I surprise myself more than once. writing in a way that is like, shit, like I am doing this like counterpoint that it's exactly how AI does it. And I don't, I hate it. And all paragraphs have the same amount of lines because it's uniform and structured. And you can see all the logic changing that people don't do. People don't write that well changed like GPT does. That's all style. What worries me about this is like, actually like writing is thinking. Like if you write something really, really long, you did learning through that. So many people say like, oh, I want to publish this idea. Like the only thing that is holding it is writing the paper. Well, actually writing the paper, it's hard because you're going to spend a lot of time. It's like, well, exactly what's the core idea here? 

Speaker 2
45:44 - 45:45
How do I present it? 

Speaker 1
45:45 - 45:53
How do I map it next to the other stuff? So it's not just writing. It's like, well, that's the hard part because you need a lot of stuff and a lot of thinking. Yes, yes. 

Speaker 2
45:54 - 45:54
And I don't see... 

Speaker 1
45:54 - 47:29
Writing down is the hard part, yeah. It gets us back to the solvers because you can write your problem down and it has a solution. Or it doesn't. You know it doesn't have a solution, but either way you know about it. People are writing less, and I can see junior developers relying too much on AI. And AI, it makes you lazy. I had a very weird experience last year. I was using AI a lot, and then I started looking at the fridge and just wanting it to open, right? Because you were like thinking, and the way you handle AI, you think something and you write it and it corresponds to what you think and it looks magical. I started looking at the fridge and imagined it would open. How crazy is that? Yes. Your brain starts getting used to thinking things and making it happen like magically because you think of the code it writes the code for you didn't do it and your brain starts putting you to achieve things without doing them. It's crazy. Yeah. So thank you very much, Rodrigo. I enjoyed this a lot. I just said a sentence that I really liked that it's this idea of the hard part is to write the spec but nowadays we live in a world If you have a spec, you have the problem solved. Thanks to like... I think so. More than ever, yeah. Because it's easier to reach to the tools that solve problems like solvers. It's much easier today. Excellent. So yeah, I think that the bottleneck is people actually like learning about these tools and start using them. Yes, yes. Like learn to solve problems and writing them down is very important. I think people nowadays, there's an anti-trend today. I call it anti-trend. People trying to get into tech and skewing like formal education because, but you get that thing from education. You need to have a repertoire of problems to draw from 

Speaker 2
47:29 - 47:30
when you face novel problems. 

Speaker 1
47:31 - 47:40
Yes, it truly helps. Perfect, Rodrigo. So thank you very much and talk to you in a few months. Nice. It was a great pleasure meeting you and talking to you. I had fun.