Hear Me Out ദ്ദി ˉ͈̀꒳ˉ͈́ )✧

Prompt engineering basics

There's a lot of people I run into that don't understand how to use an LLM effectively and don't know prompt engineering basics. Either they are too intimidated to use it or feed it prompts like "Tell me the news related to xyz," which is too vague, or they think that they have to write a full paper every time they're authoring a prompt, which is not the case.

Prompt engineering is not as hard as it sounds and you can have conversations with the LLM in such a way that it feels more natural. I'll go over some tips with examples. I'll also have some vague recommendations for private LLMs I would recommend near the end.

ℹ️ Disclaimer up front that I am not an expert, your mileage may vary, these are personal tips and do not represent myself or where I work, etc. etc. etc.

Important things to include

Primarily, in my opinion, three things are critical for accuracy with generative AI models:

  1. Context
  2. Goal(s)
  3. Grounding

Context

The AI needs to have full information to generate the most accurate response possible. If you, say, ask it a question like, "I'm on Linux. How do I install an appimage?" You're going to get a wide range of results because the AI lacks key information. It doesn't know:

Goal(s)

The AI needs to know what your goals are in order to frame how it responds. What question do you want answered? How do you want the answer, if that matters to you (in paragraphs? bullets? markdown formatting?)

Grounding

Generative AIs are known for inaccuracy and hallucination. You can prevent this as much as possible by giving the model guidelines within which to operate - e.g. telling it to research, confining it only to the current conversation (having it disregard its memory of other conversations), asking for citations, telling it if it cannot find an exact source to not make something up.

Examples

Example 1

Here's an example related to my computer from when I was first moving to Linux:

I have an AMD computer running Zorin OS (Ubuntu based Gnome on Xorg). My current specs are below. Two of my storage drives (the 4 TB drives) are NFTS formatted from my old Windows install and they're not cooperating with my new system or Steam. I need to copy the data off of them onto a new drive so that I can reformat them with the proper system format. I'm wondering what the best option is to do that - I'm thinking about buying another drive, one with much more memory possibly, and making that my primary drive on my computer so that my mounted drives don't serve as my main repositories any longer. What type of drives should I get that will be the fastest and would be compatible with my computer? Should I get another NVME drive? Would my existing NVME drive still be able to work with the set-up as it is currently? Please look up the parts in the list for more information if needed. [Computer specs]

So note here:

  1. Context: full context about the system is provided so it has every parameter it needs for a proper recommendation (AMD computer, Zorin OS, Ubuntu base on Xorg, NFTS drives that are NVME, my exact computer part specs)
  2. Goal: provided to the model in the form of questions to be answered so it isn't open ended
  3. Grounding: "look up the parts in the list for more information if needed" (instructing it to research)

Example 2

Let's say you want to run a series of hypotheticals about the future through the model. Here's what that could look like:

I want to run a complex scenario by you for an assessment on what could happen and what could be done to stop the events as they are unfolding in [country]. I am going to outline the events in a series of points that are sequential in order of when they happened. Please do not use any of our other conversations for context; only the new information in this one. If you understand, let me know and I will outline the scenario.

  1. Context: will be provided once the model confirms it understands the goal
  2. Goal: give me a scenario assessment with future projection and what could be done to stop it from going further
  3. Grounding: "please do not use any of our other conversations for context" so that it is not influenced by any possible bias I may have introduced; LLMs are incredibly word sensitive since that's how generative AI operates, so bias is very easy to inject unknowingly just through semantics.

Private LLMs

To get this out of the way quickly: in my personal opinion there are no truly private or anonymous LLMs unless you locally host.

Before you leave after reading that, it's not as hard as you think.

It can be as easy as installing the program and picking the model.

If you want to try running locally, I recommend Ollama though it's probably easier to just do it through something like kDrive and let that handle the configuration for you.

Here's a good video on how to get started with Ollama by Tech with Tim: link.

If you decide to use something like kDrive, you can choose their AI tools at a decently low price threshold and they handle configuration and manage things for you: kDrive AI Tool Prices

I'm planning on writing a full review of kDrive soon as I've been testing a lot of different European Cloud alternatives to Google.

There aren't really any other options for privacy when it comes to LLMs, though I think the closest you can try to get is Mistral's Le-Chat. As they are based in France, they have to follow European data guidelines, which gives you an added level of security that you just won't get from an American based LLM. If you have a student email, it looks like you can get their Pro version at the time of writing for $5.99 per month: pricing for le-chat. Bonus: their logo doesn't look like part of someone's nether anatomy.