New Resources for Teaching with Legal AI and Keeping Up with the Latest Research

Today’s guest post comes from the University of Denver Sturm College of Law’s John Bliss. Professor Bliss has been kind enough to share some resources that he has crafted to help teach AI to lawyers and law students. In addition, he has launched a new blog which would likely be of interest to our audience so we are happy to host this cross-over event.

Teaching

I recently posted to SSRN a forthcoming article on Teaching Law in the Age of Generative AI, which draws from early experiments with AI-integrated law teaching, surveys of law students and faculty, and the vast new literature on teaching with generative AI across educational contexts. I outline a set of considerations to weigh when deciding how to incorporate this tech in the legal curriculum. And I suggest classroom exercises, assignments, and policies. See https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4682456.

Blog

I’m also organizing a blog that provides up-to-date analysis of research on AI’s legal capabilities. You can subscribe at http://ai-lawyering.blog. Let me know if you’re interested in contributing. The motivation for the blog is that legal AI is a fast-moving field. It is all too common that our discussions are based on outdated and inaccurate information. Empirical findings are often misconstrued in mass and social media. The blog aims to address this issue by reviewing the latest high-quality research, emphasizing empirical studies of AI capabilities as well as scholarship on the implications of this technology for lawyers and other stakeholders in the legal system.

New Program on Generative AI for Legal Practitioners

I’m working with a non-profit teaching lawyers and other members of the legal profession about generative AI: http://oak-academy.org. Just last week, we held our first session with a group of lawyers, law students, and academics. It seemed to go well!

I look forward to continuing conversations on these topics. Please feel free to reach out—jbliss@law.du.edu

Review: vLex’s Vincent AI

Vincent is vLex’s response to implementing AI into legal research and it’s the most impressive one that I’ve seen for legal research.  Damien Riehl was kind enough to give us a personalized demonstration (thanks for setting that up, Jenny!) and it was a real treat to be able to ask questions about it in real-time.  I would say that the best way to see this in action is to schedule a demo for yourself but if you want to hear my hot-takes about the platform, please keep reading. 

Vincent is Really Cool 

Interface 

Many times when you engage with these models they feel like a complete black-box.  You put in some text, 🪄 presto-chango 🪄, and then it spits something back to you that seems related to what you put into it.  Vincent instead offers you a fairly controlled interface that is centered around what you typically need for something like real-world legal research.  That’s because this doesn’t look like a “chatbot,” sandbox-type experience and feels more like a tool that a professional would use. 

You Can Tell Where It Gets the Information

This is huge because almost everything you need is on one page immediately.  You ask it to draft a legal research memo and the cases are just to the right of the memo.  The relevant portions of the cases have been summarized and presented there for you.  A tool tells you how confident Vincent is that this is close to your request.  Everything below 70% is dropped.  You can toggle between cases, regs, statutes, and secondary materials available.  Everything that could require a deeper dive has a hyperlink.  You can get a sense of what this looks like from vLex’s website about Vincent here: https://vlex.com/vincent-ai.  

Multi-Stage Prompting 

vLex is probably best known for its deep archive of primary international materials.  Vincent uses this to great effect (since we know that many of these NLP technologies started as translation tools).  You can enter a natural language question in English, Vincent will translate it, run the search in the home country’s language, and then provide you with both the original text (so you could translate it yourself) and an English (or whatever) language translation.  Sexy stuff for you FCIL researchers. Also, this is substantially more powerful than something that simply tries to grind through many iterations of similar keyword searches in other languages.   

It’s also a noteable example of multistage prompting and retrieval in legal research.  You can see that it is being fed through not one prompt but many complex chains to produce high-quality, useful output. The tools for US caselaw are similar: Your query is turned into several different prompts that run off in different directions through the vLex database to retrieve information. Some prompts are searching through cases, statutes, regs and their secondary materials to see what is useful; others might be summarizing cases as they relate to your query; other prompts are finding counterarguments; another prompt is evaluating them for confidence on the your specific subject etc. etc. and a final prompt is summarizing all of this information into a neat little report for you. In summary, they’re making great use of the technology’s potential by deploying it in many different ways. The final product is sort of a fabricated, personalized secondary source created by running tons of prompts over the underlying primary materials. In fact, Damien calls this a “Me-tise” 😂 (apologies to Damien if I stole his punchline) and he foresees it being a powerful new tool for legal researchers. I’ve been bullish on the fabrication of secondary materials since I first saw what these things could do so it was exciting to see a precursor of this in action. 

Damien let us know that behind the scenes they are using a combination of the various LLM’s to achieve these results and cut costs when possible: Claude, Llama2 (Meta), and GPT4. We met with him shortly after the OpenAI controversy and he pointed out that they are able to swap models in vLex if necessary.

Secondary Materials and Market Share 

We have all come to love and rely on specific secondary materials that exist in Westlaw and Lexis. vLex’s acquisition of Fastcase meant that they acquired a huge, fantastic database of primary US materials. The one pain point for people who may be interested in switching from Westlaw/Lexis to Fastcase was the relative dearth of secondary materials available. The features that I saw last week in vLex may fill that need for some users and it will be interesting to see if people are lured away from their favorite practice guide or treatise published by Lexis or Thomson Reuters because a robot can now do some of that work summarizing and analyzing vast quantities of primary law. It will also be interesting to see if Lexis and Westlaw will roll out these types of features, since they could be in direct competition with their robust (and pricey) secondary materials offerings.

Before I get a slew of angry emails: I recognize that a traditional secondary material does much more than summarize cases, statutes, and regulations but it does some of that (also remember we’re still in the infancy of this technology for legal research). If that is all the researcher needs, then these tools could work as a replacement for some people (and they don’t rely on monthly updates – they do this on demand). That may allow some people to cut ties from Lexis and Westlaw in a way that could shake up the industry in a way that disrupts the status quo. It could also be incredibly powerful for something like a 50-state survey or even surveys across many different countries. Feel free to let me know what an ignoramus I am in the comments if I am missing something here.

Outstanding Questions 

Price 

I’ll dive right in where you all have questions, “Can we afford this thing?”  Dunno and it depends (super satisfying, I know).  The difficulty here is that these things are still very expensive to operate.  The more sophisticated the model, the larger the database, the more complex the stages of prompting, the various modalities (scanning documents, reading the screen, etc.) – the more it costs them.  They are all trying to figure out how to create a pricing structure where they can 1) offer it to the widest audience possible and 2) remain profitable.  As we know, their primary source of revenue is the big firms and so the product is currently only available in paid beta for select companies. 

Damien and vLex are both refreshingly upfront and clear about this.  No hand-waving or sales talk, which I think is why so many people in our industry look to people like Damien for information about these technologies as they are developed.  Damien mentioned that they are taking the “democratize the law” call to action from Fastcase seriously and are looking for ways to make it affordable on the academic market.  

Possible Future Options 

This is all complete speculation on my part but some sort of limited version of the platform seems like it could be reasonable for the academic market (like BLaw does with their dockets): limited uses per day, limited uses per account, a “lesser” account with limited features, etc. As the market stands today academic law libraries have access to a limited version of Lexis AI, trial access to Casetext Cocounsel (unless you’re willing to pay), no access to Westlaw Copilot, no access to Harvey AI, and no access to vLex. I anticipate all of that will change as the prices come down. The point of frustration is obviously that we want to be able to evaluate these tools so that we can teach them to students, in addition to using them ourselves so that we can benefit from the technology.

In conclusion, Vincent by vLex represents a significant step forward in AI-driven legal research. Its sophisticated multi-stage prompting, transparent sourcing, and potential in fabricating secondary materials make it a formidable tool. The future of Vincent and similar AI platforms in the academic and broader legal research community is certainly something to watch closely.

Keeping Up With Generative AI in the Law

The pace of generative AI development (and hype) over the past year has been intense, and difficult even for us experienced librarians, masters of information that we are, to follow. Not only is there a constant stream of new products, but also new academic papers, blog posts, newsletters, and more, from people evaluating, experimenting with, and critiquing those products. With that in mind, I’m sharing my favorites, and I’ll also pepper in a few recommendations from my co-bloggers.

Twitter

Before Twitter began its slow decline, it was one of my primary sources for professional connection, and there are many there who are exploring generative AI. I especially enjoy following people outside of the legal world. Many of my favorites are still there, like Ethan Mollick, Anna Mills, and Lance Eaton (all in higher education) as well as critical AI theorists like Timnit Gibru and Emily Bender.

LinkedIn

Despite the good bits that remain on Twitter, many interesting legal tech discussions seem to have moved to LinkedIn (or perhaps I’ve only recently found them there). Some of my favorites to follow on LinkedIn (in no particular order beyond how I’m running across them as I scroll) are: Nicole Black, Sam Harden, Alex Smith, Cat Moon, Damien Riehl, Dennis Kennedy, Uwais Iqbal, Ivy Grey, Robert Ambrogi, Cat Casey, Nicola Shaver, Adam Ziegler, and Michael Bommarito. Both Bob Ambrogi and Nicola Shaver recently had posts gathering legal tech luminaries to follow, so I would recommend checking out those posts and the comments to find more interesting folks. And if anyone else has figured out the LinkedIn etiquette for connecting vs. following someone you only know via other social media, please let me know.

Newsletters

Most of us have many (many, many) newsletters filling our inbox each day. Here are some favorites.

Jenny:

  • AI in Education – a Google group
  • Lawyer Ex Machina – from law librarian Eli Edwards, on legal technology, law practice and selected issues around big data, artificial intelligence, blockchain, social media and more affecting both the substance and the business of law (weekly)
  • The Neuron – AI news, tools, and how-to
  • The Brainyacts – from Josh Kubicki, insight & tips on generative AI use in legal services (daily)

Rebecca:

  • One Useful Thing – from Ethan Mollick, mostly on AI in higher ed (weekly)
  • Do Something – from Sam Harden, on legal tech, often from a small firm and access to justice angle
  • Legal Tech Trends – legal tech links, podcast, articles, products, along with original pieces (every two weeks or so)
  • KnowItAALL – this daily newsletters is a benefit for members of AALL (American Association of Law Libraries), but it is also available to non-members for a fee; great coverage of legal AI, I read it every day
  • AI Law Librarians – is it gauche to recommend our own blog? You can subscribe as a newsletter if you like!

Sean:

Podcasts

There are loads of podcasts on AI, but here are a few we follow:

Blogs & Websites

We’re bloggers, we like blogs. Traditional media can be ok, too, although mind the paywall.

YouTube

Sean also mentioned that much of the interesting stuff is on YouTube, but that it is fairly high-effort because many of the videos are an hour long, or more. Maybe we’ll convince him to share some of his favorite videos soon in a future post!

A Few LibGuides

If you still need more, here are a few libguides:

What about you?

Who are your favorites to follow on social media? Are there helpful newsletters, blogs, podcasts, or anything else that we’ve missed? Let us know in the comments.

A Blog Was Born (Or A Blog Arises from the Ashes of Forsaken Blogs)

Welcome to the inaugural post of AI Law Librarians! This blog aims to delve into the intersections of generative AI, law librarianship, legal research, legal technology, legal education, teaching legal research, and perhaps other TBD topics.

The origin story of this blog is a tale as old as time – good intentions smacked up against reality and inertia. Excited about our new chatty chatbot friends that began proliferating at the end of 2022, all of us (Sarah and Sean individually, and Jenny, Becka, and Rebecca collectively) thought to ourselves, “Self (“selves” in the case of the trio), I/we should start a blog about generative AI from the law librarian perspective! The initial enthusiasm was real – we each jumped on creating sites or/and snapping up domain names. However, we each soon realized it would be a lot of work, and our initial efforts never really took flight, in fact, the fledgling fell out of the nest into a caldera of molten lava – until now!

Sarah

My own little slice of the story is that, after several years as a RIPS Law Librarian Blog contributor, I was suddenly out of things to say, so I resigned in 2021. However, when ChatGPT-3.5 hit the scene at the end of 2022, I was so taken with the new technology that I started to have, like…thoughts again and began to barrage the editor with unsolicited ChatGPT guest posts.

Finally, after years of anticipation harking back to the days of The Flintstones, my robot companion had finally arrived—or at least a version of it. And as the excitement/apprehension in the media grew, it became evident that interest in generative AI was going to survive beyond the usual hype cycle. This new technology wasn’t merely a chatbot for amusement, it was poised to remake (erm… or destroy) the world as we know it and revolutionize everything, including law practice and education. Suspecting that the RIPS Law Librarian Blog editor was tiring of my unsolicited musings, I considered starting my own blog and even went as far as creating a WordPress site, only to quickly abandon the idea after deciding it would be too much work.

I will leave Sean his own slice of the origin story, but we had previously discussed the lack of outlets for law librarian blog posts about ChatGPT et al. After reading a particularly informative guest blog post by him on the RIPS-SIS blog entitled The Case for LLM Optimism, I emailed to throw out the idea of starting our own blog. To my mild surprise, he readily agreed (sun devils and wildcats unite!), and we searched our minds for a domain name and some like-minded folks to join us on this unprofitable adventure, and shortly thereafter, AI Law Librarians was born.

Sean

In my journey to the blog, I recently penned a couple of articles I posted on SSRN about fine-tuning LLMs for legal practice and the innovative use of chatbots by law students, and a couple of blog posts which had no obvious home in law library publications. Like Sarah, I didn’t want to swamp the RIPS-SIS blog editor with guest posts, and I had the idea of starting my old blog. As I was planning to escape the Arizona desert for a new job as the Director of Technology Innovation in Oklahoma, Sarah emailed me and suggested we start our own blog, and I was like, “Let’s get on Zoom today!” The brainstorming for domain names was brief—Sarah had a few ideas, and then “ailawlibrarians.com” popped into our minds. We pondered who to ask to join our endeavor for a few days before deciding on Rebecca, who had seemed interested in AI at the LIT-SIS roundtable in Boston.

Rebecca

I have been scrambling since November 2022 to understand this new technology and how it will affect legal research and writing. Several months ago, I connected with Jenny and Becka, and we have presented several times over the summer (together and separately) on AI and the law, mostly to faculty and attorneys. Then Sarah and Sean approached me just as the summer was coming to a conclusion and I realized that despite wanted to write on these topics these summer, I hadn’t actually *published* anything. I had a few blog posts in the hopper, so it was perfect timing. I shared with them that a couple of librarian colleagues, Jenny and Becka, and I had begun our own blog venture with the name chatgptandfriends.com, which was currently stalled after a “Hello World” WordPress post. But then, an idea emerged – why don’t the five of us combine efforts?  Sarah and Sean were excited by the prospect of getting three people by asking just one, and Jenny and Becka agreed. 

Becka

I had taught AI and the Law twice when generative AI became something that everyone knew and was worried about. The more I read about generative AI, the more concerned I became – not because I was worried about academic misconduct or cheating, but because I was worried that law faculty and law librarians would swing the teaching pendulum so far back in the other direction that it would harm some of our most vulnerable students. There were legal issues, ethics issues, pedagogy issues, and places where AI intersected with the other courses I teach: Education Law and Disability Law. Jenny and I talked about how to get the word about the learning positive and negative uses of AI out there and then Jenny brought in Rebecca. The three of us have been talking for months and I’ve also been talking with my university’s committee on generative AI and the law school’s. When Rebecca heard from Sarah, it made sense for all of us to get together.

Jenny

As Becka noted, I like generative AI.  My origin story is not nearly as exciting as some…”I teach law practice technology, so I better check this thing out…well, this is going to cause a fuss when people don’t realize it hallucinates!”