09/28/24: Hacking an LLM Using the Z3 Theorem Prover with Ian Bicking

Navid Hashemi recently defended his PhD at USC and is about to begin a post-doc at Vanderbilt.  His research focuses on the intersection of Artificial Intelligence and Temporal Logics, with applications in Formal Verification of Learning Enabled Systems and Neurosymbolic Reinforcement Learning.  Today Navid joined us for a really exciting presentation about his work on metrizable logics for reinforcement learning, and a technique for verification thereof based on the over-approximation of reachable sets using ReLU.

Chengpeng Wangworking with Prof. Xiangyu Zhang. His research focuses on program analysis, especially software analysis, and in particular how existing analysis techniques intersect with emerging approaches from AI such as Large Language Models. Today Chengpeng joined us to talk about his recent NeurIPS paper proposing a novel static analysis technique based on LLMs. The technique is very interesting and highly informed by prior works in the static analysis space, but leverages LLMs as a kind of "oracle" to solve problems which, when handled statically, quickly become untenable. This was a really interesting talk and we're very greatful that Chengpeng took time out of his Sunday afternoon to talk to us!

Harry Eldridge is a Cryptography PhD student at Johns Hopkins, advised by Abhishek Jain and Matthew Green. His research (so far) touches on security and privacy implications of commodity hardware, which is a fascinating topic deserving of the mathematically disciplined, cryptographically informed approach his lab takes to such problems. Today Harry joined us to talk about his research into the problem of AirTag stalking, and how it can be ameliorated, while retaining acceptable performance, through cryptographic protocols. This was a very interesting talk with serious, real-world implications, and we hope you enjoy it as much as we did!

Ian Bicking is an engineer at Brilliant, which is also what he is. (Sorry, dad joke). Ian joined us today to talk about his super charming (and extremely interesting) weekend of experiments hacking various LLMs to solve puzzles using z3. The presentation was roughly the first 2/3 of the event and the remaining third presented a fantastic conversation about the future of AI, tool use, chain and tree of thought, o1, and more. Thanks again for joining us Ian!

Dhekra Mahmoudat LIMOS in Clermont-Ferrand, France, where she researches the formal analysis of cryptographic protocols under the supervision of Pascal Lafoucade and Jannik Dreier. Today Dhekra joined us to present her recent USENIX paper Shaken, not Stirred -- Automated Discovery of Subtle Attacks on Protocols using Mix-Nets. This was a really interesting presentation with a good conversation afterword touching on some subtler points around the Dolev-Yao threat model, the limitations of ProVerif, and proof optimization.

Matej Panciak holds a PhD in mathematics and is a software engineer at the Argument Computer Corporation, where among other things, he works on Lurk. Lurk is a LISP for defining computations that can prove (in the ZKP sense) that they ran, which is probably useful for all sorts of cool things we haven't thought of yet, but right now, is pretty important for doing stuff on-chain. (I can easily imagine this being applicable to building something like a dweb version of AWS ... in some theoretical future where FHE is so good that you can just trust randos to run code for you). Anyway, Matej presented a super rad intro to Lurk, gave us a code demo (it worked!) and then enjoyed our usual über-nerd conversational segment at the end.

Derek Egolf is a PhD student (since 2021) at Northeastern University, advised by Stavros Tripakis. His primary research focus is the automatic generation of correct-by-construction systems from high-level specifications (synthesis). Today Derek talked about his recent paper in this vein, Efficient Synthesis of Symbolic Distributed Protocols by Sketching, to appear in FMCAD. This was a very interesting talk with a technical conversation afterword. We hope you enjoy it as much as we did!

Joshua Ramette (https://x.com/RametteJoshua) recently completed a PhD in physics at Mass Tech, and today he joined us to talk about his new project, Undermind. Josh and his friend Tom Hartke (https://www.tomhartke.com/) founded Undermind (YC S24) to radically improve academic literature search using a mixture of AI techniques. Their system is slow, deliberate, and very high quality. You can check out Undermind at www.undermind.ai , or peruse the query I did during the Q&A section here: https://www.undermind.ai/query_app/display_one_search/c743b66ee4378b12ae8bad1fe58975ba95da71e7f7a1d5c2f0c6973c677648cc/

Evan Pu ( https://evanthebouncy.github.io/ , @evanthebouncy on X ) is a senior research scientist at Autodesk AI Lab, working on code-generation for human-machine collaboration in CAD, and industry scale instruction-following dataset annotation. Today Evan joined us from a toilet (with the lid closed) so as not to wake up his wife due to a rather large time-zone delta, which was hilarious and a first for the Boston Computation Club. Anyway, this was a really fun talk with excellent Q&A and we hope you enjoy it as much as we did!

Arthur O’Dwyer is a C++ programmer and blogger who today joined us to talk about his musings on the algebraic structure of the popular web-game Infinite Craft. Infinite Craft is a clever little experiment in sandboxed exploration, and it turns out to give rise to a rather complex mathematical structure with some interesting background in theoretical CS. Arthur covered all this and more in his presentation, which was super interesting and a lot of fun to watch. Check out Arthur's original blog post here: https://quuxplusone.github.io/blog/2024/03/03/infinite-craft-theory/ Check out Arthur's slides here: https://bstn.cc/artifacts/arthurODwyer/infiniteCraft.pdf