I hope you have enjoyed the previous blog where we discussed step by step how to chat to our PDF using RAG approach.

We will be continuing that code and modifying it to get more accurate results.

Introduction to Query Transformation

The entire game is of getting relevant answer to the user, getting what the user not just ‘asked‘, but what the user intends to ask.

We are already aware of Google search, do we get only what we have searched for, or all the things related and relevant to that query?

User’s query is not accurate enough to get what he actually needs

It has to be polished enough to get that specific thing from the AI

User’s Query - can be more abstract and can be less abstract as well

Ek example -

Let’s say aap ki ek dukaan hai hardware ki, waha pe sirf aap ku pata hai ki konsi chiz kidhar hai and kya naam hai etc etc

Ab aap ko zarurat padgayi ki ek ladke ko dukaan par rakhe - gaahak bahut zyada hai, dukaan badi hai

Ab usko kuch kuch chize malum hai, lekin experience dukaan ka nahi hai

Isliye wo pehle din se sahi kaam nahi karega, galtiya karega, ek maango koi aur chiz laayega,

isko pehle se hi samjhana hoga ki gaahak kya kahega - gaahak shyd kuch aur puche, gaahak ko shyd na maloom ho us chiz ke bare me, gaahak galti se kuch aur puch le

Ab aapko ye sab train karna hoga so that apka ladka jab aap na ho dukaan par, jab bhi wo usko sambhal sake

Aise hi RAGs ka masla hai, they are intelligent, but still we need to polish them and make them reiterate on what user has asked and what to respond to him…

Parallel Query - Fan out retrieval

Let’s explore how this approach works

So, here we will be using langchain package

First, we have to import Multi Query Retriever

from langchain.retrievers.multi_query import MultiQueryRetriever
from langchain_openai import ChatOpenAI

Then, we need to call this Multi Query retriever on the regular retriever we have

Before that, we need multiple queries generated from the user query, so we are using ChatOpenAI model to perform that task

llm = ChatOpenAI(
    model_name="gpt-4o-mini",
    temperature=0,
    openai_api_key=os.environ.get("OPENAI_API_KEY")  
)

Next, we will be using multi query retriever to do multiple similarity searches and get relevant chunks from them and then we can derive the context for our AI model

retriever = QdrantVectorStore.from_existing_collection(
    url="http://localhost:6333",
    collection_name="langchain_learning",
    embedding=embedder
)

multi_query_retriever = MultiQueryRetriever.from_llm(
    retriever=retriever.as_retriever(),
    llm=llm
)

Final step is to set the context and log our output, here it internally selects unique chunks from multiple results and returns that

relevant_docs = multi_query_retriever.invoke(user_query)

SYSTEM_PROMPT = f"""
    You are a helpful AI assistant who has access to a specific document of user,
    and user will ask questions from it, answer only those,
    i have given you context from where you have to answer it

    context={relevant_docs}
"""

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_query},
    ]
)

print(response.choices[0].message.content)

The output i got for the question “Explain the major challenges faced during AI model deployment.“

The major challenges faced during AI model deployment include:

1. Adjusting Workflows: Developers need to adapt their existing workflows, prompts, and data to work with the new models, which can have unique quirks, strengths, and weaknesses.

2. Versioning and Evaluation Infrastructure: Without proper infrastructure in place for versioning and monitoring the performance of the models, deployment can lead to numerous complications and operational headaches.

3. Regulatory Changes: Regulations surrounding AI technologies are constantly evolving. For example, AI resources can be heavily regulated as national security issues, and compliance with regulations such as the GDPR can be costly and complex.

4. Compute Resource Availability: Changes in laws can suddenly limit access to compute resources, such as being banned from purchasing GPUs from certain vendors, impacting the ability to deploy models effectively.

5. Intellectual Property Concerns: There are uncertainties regarding intellectual property when utilizing models trained on data that may not be owned by the developer. This can create hesitancies, especially for companies deeply invested in their IP.

These challenges highlight the importance of thorough planning and consideration of the evolving landscape of AI deployment.

Stay tuned, as we will discuss more approaches in Query Transformation in our upcoming blogs

• Reciprocate Rank Fusion

• Query Decomposition

Well, this is the end of one of the concepts of Advanced Rag approaches - Parallel Query Retrieval.

See you in the next blog

Let’s connect here on Twitter

Peace out ✌️

This is your friend Mubashir

Today we’ll be discussing an interesting thing that is RAG…

I have covered all the steps in detail, open your vs code or any IDE you use, and let’s code together

Rerieval-Augmented Generation — an AI technique that combines the power of retrieving information using LLMs, so that accuracy and context-awareness of AI models can be enhanced.

Getting relevant data to the user according to their needs is a difficult thing

1. Indexing → Data source - chunking - Vector embeddings - stored in vector DB

2. User asks a query → make vector embeddings - search in vector DB - we get relevant chunks - filter out the chunks - AI being called (with relevant chunks and user query)

3. Implementation- Langchain

Let’s make a RAG on PDFs - in simple terms, let’s talk to our pdfs…

first step is to install pdfloader

pip install langchain_community pypdf

Let’s first start by loading a PDF

So, we need to give path to our loader - here is a dynamic way to give path using pathlib library.

After calling load() function - print what the loader has, and you will see the contents of your pdf

from langchain_community.document_loaders import PyPDFLoader
from pathlib import Path

file_path = Path(__file__).parent / "AI_Engineering.pdf"
loader = PyPDFLoader(file_path)

docs = loader.load()

print(docs[50])

Now it’s time for splitting the text

pip install langchain_text_splitters

There is one issue still, while splitting - which one is correct fit for us
For PDFs we need RecursiveCharacterTextSplitter

from langchain_text_splitters import RecursiveCharacterTextSplitter

Now, we’ll call the text splitter, we need to also mention how many chunks we need and for better context, do we need to have overlaps between chunks?

text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)

Once we have set the number if chunks and overlaps, it is time for applying it on our document

split_docs = text_splitter.split_documents(documents=docs)

Chunking part is done

Now let’s move on with “Embeddings“

Let’s install langchain’s openAI embedding model

pip install langchain-openai

Now import it

from langchain_openai import OpenAIEmbeddings

Let’s call the embedding model and specify a model and our API_KEY

embedder = OpenAIEmbeddings(
    model="text-embedding-3-large",
    api_key="YOUR_API_KEY"
)

Now it is time to embed our document as vectors and store it in a DB, not a regular DB but a vector database.

We’ll go with Qdrant DB

Let’s install it

pip install qdrant-client

After this, we have to setup DB on Docker, so we’ll create a docker-compose.db.yml file

Inside that file we have to write the below code

services:
  qdrant:
    image: qdrant/qdrant
    ports:
      - 6333:6333

To run this docker file, paste the below command

docker compose -f docker-compose.db.yml up

Done

Now it’s time to install langchain-qdrant

pip install langchain-qdrant

Import Qdrant client and vector store in our code now

from qdrant_client import QdrantClient
from langchain_qdrant import QdrantVectorStore

Create a collection now, it needs url of Qdrant db, collection name and the embedder.

This documents parameter will create a collection for us, if there are no collections in vector DB

vector_store = QdrantVectorStore.from_documents(
    documents=[],
    url="http://localhost:6333",
    collection_name="langchain_learning",
    embedding=embedder
)

Also we need to add those split_docs to our vector_store

vector_store.add_documents(documents=split_docs)

Now run the code and checkout this url

http://localhost:6333/dashboard

Once you see the injection is done, you can check the dashboard

Now, comment that specific part of storing the data in vector DB as we are done with it

# vector_store = QdrantVectorStore.from_documents(
#     documents=[],
#     url="http://localhost:6333",
#     collection_name="langchain_learning",
#     embedding=embedder
# )

# vector_store.add_documents(documents=split_docs)

This same we have to do now, but for retrieving the data from the DB

retriever = QdrantVectorStore.from_existing_collection(
    url="http://localhost:6333",
    collection_name="langchain_learning",
    embedding=embedder
)

Now, let’s do the main thing, getting the data from the DB for which we were writing these many lines of code

relevant_chunks =retriever.similarity_search(
    query="What are vector embeddings?"
)

Done, now just we have to set up the OpenAI’s response for user’s queries (last and the most important part)

Install OpenAI

pip install openai

import it and write a system prompt for it also include our final output - ‘relevant_chunks‘ variable

from openai import OpenAI
client = OpenAI()

SYSTEM_PROMPT = f"""
    You are a helpful AI assistant who has access to a specific document of user,
    and user will ask questions from it, answer only those,
    i have given you context from where you have to answer it

    context={relevant_chunks}
"""

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_query},
    ]
)

print(response.choices[0].message.content)

boom, you’re done

let’s look at the output for our query

Hope you’ve enjoyed this, let me know in the comments.

Let’s connect on twitter for further discussions

Peace out ✌️

Welcome to the GenAI blogs series

Today we are discussing prompt engineering, how to draft a better prompt so that we can utilize AI the best way possible.

What is a prompt??

An instruction given to the model, technically they are the initial tokens provided to the model.

Alpaca prompt - “Self instruct method“

Alpaca (Llama-7B) is a model that was trained by meta using self-instruct method - it includes three things → instruction, input and output

Instruction: {instruction}
Input: {input}
Response:

inst format - Instruction tuned models

Models trained like this are usually to behave more like helpful assistants rather than just predicting the next word.

<s>[INST] <<SYS>>
{{ system_prompt }}
<</SYS>>

{{ user_message }} [/INST]

• The INST part is filled with instructions that the model has to follow.

• System prompt is wrapped inside «SYS»…«/SYS» tags, where the model is assigned with a role.

• And finally the user query or message is written.

<s>[INST] <<SYS>>
You are a helpful and concise AI assistant.
<</SYS>>

Explain the difference between supervised and unsupervised learning in simple terms. [/INST]

chat ml openai format

This is the most significant thing, it is used in chatGPT, which is made by openAI.

<|system|>
You are a helpful assistant.
<|user|>
Explain gravity in simple terms.
<|assistant|>
Gravity is a force that pulls objects toward each other...

So, if you have ever tried building upon openAI’s api, then you have definitely used this

[
  {"role": "system", "content": "You are a helpful assistant."},
  {"role": "user", "content": "What is gravity?"},
  {"role": "assistant", "content": "Gravity is..."}
]

In short, it is a clean, extensible, and readable way for conversations.

Prompting styles

In a prompt, there are majorly two parts - Instruction and context

So based on this we have few different styles of prompting

Zero shot prompting

In this style - we straight away ask the AI our queries - no context nothing - solely trusting AI’s pre-trained knowledge

Translate the following sentence to French:
"The cat is on the roof."

# it gives the answer as expected
Le chat est sur le toit.

Few shot prompting

On contrary, here along with the instruction we provide few examples to the model

Translate English to French:

English: The sun is shining.  
French: Le soleil brille.

English: I love chocolate.  
French: J'aime le chocolat.

English: The cat is on the roof.  
French:

This style is used when the output is ambiguous or has some hidden knack in it etc

What is system prompt - sets initial context, so that it can be controlled by developer from getting manipulated by the user

Chain of thought prompting

This is the style which even we follow while we solve math problems, like we used to do in high school

So, when we solve step by step then it is less error prone.

I still remember in my 11th standard (inter 1st year) final exams, i lost my marks in simple pythagorean problem just due to skipping 2 steps and landing into a wrong solution.

The same applies here, when AI model addresses any query step by step, it can perform better and be less erroneous.

Let’s take a math question and solve it using AI model using this specific approach

Prompt: "If a train travels 60 km in 1.5 hours,
then continues at twice the speed for another 2 hours,
how far did it go in total? Let’s solve this step by step."

AI’s Chain-of-Thought Response: 

First, calculate the speed of the train in the first part: 
Speed = 60 km / 1.5 hours = 40 km/h 

In the second part, the train goes at twice the speed: 2 × 40 = 80 km/h

It travels at 80 km/h for 2 hours: 80 × 2 = 160 km 

Total distance = First part + Second part = 60 + 160 = 220 km

Final Answer: 220 km

Self consistency prompting

This builds on Chain of Thought prompting, but more smarter than previous method.

Instead of trusting the model’s first chain of thought, we ask it to generate multiple reasoning paths and then take a majority vote on the answer.

Because sometimes the model gets it right only after a few tries. Picking the most frequent or consistent output improves accuracy.

How it works:

• Ask the same question multiple times using CoT prompting.

• Collect multiple outputs.

• Choose the most common or reasonable answer.

It’s like brainstorming with yourself and going with the most agreed-upon solution

Role based prompting

More than just the above styles, AI model can disguise itself as someone else’s personality or say takes up a role.

This is the most general thing we have done in projects and daily prompting i believe.

Act as a linux admin, 
You are an intelligent AI model
You're a math teacher...

Contextual prompting

This style is specifically used in personalized chatbots, say customer help support bots, where the bot has to be aware of the company’s stuff and the services they do etc

Even when we chat with chatGPT or any other model, the chat history and previous chats matters and it helps us a lot in framing our queries.

This one also a great selling point just to keep the context safe and history of our chats.

Multi modal prompting

Modals here refer to different input/output types other than text - audio, video, images, files etc

LLMs like GPT-4-Vision, Gemini, Claude 3, or MM1 can process multiple modalities — and respond in kind.

So, once i tried giving video to gemini and extract the speech in the form of text from it - and boom, it was a banger - this is a great money making thing if properly utilized.

I hope you have enjoyed it, let me know in the comments which one do you use the most 😉

Follow me here on X/Twitter

Peace out ✌️

Your words get tokenized—broken into tiny pieces—then turned into vector embeddings that carry their meaning. These embeddings pass through encoders, layer after layer, guided by attention to focus on the right parts. They travel through a huge transformer model made of deep neural networks, trained on massive amounts of data. The AI looks at everything within its context window, uses learned weights to understand what matters, and runs softmax math to guess the next word. The decoder then builds your reply, picking from a giant vocab.

Don’t worry 😅, we’ll decode this, step by step.

Phase 1: Input and Encoding

Let’s say user gives an input to chatGPT: “What is the meaning of Artificial Intelligence?”

This input is first divided into smaller chunks that is tokens (some numbers), as we know machines don’t take input in our language, rather they convert it into numbers, so this is how it goes

Let’s see that in real time from a website called TikTokenizer

Tokens to vector embeddings

Now that the text is being converted to tokens - it’s time for vector embeddings

The main motive of vector embeddings is to get the semantic meaning of the words

Every word has its vector form…

huh, wait 😌

See, imagine you heard a sentence, so your brain processes it first right?

In this AI world, our brain is resembled as a ‘high dimensional map‘, like our brain is full of memories, emotions etc

Similarly, every token has a vector in this high dimensional world.

When we hear the word ‘APPLE‘ - we might think of a fruit / brand - apple or something else.

So, the token of Apple has a vector in AI world and this way AI understands what the user actually meant

This is the entire process of VECTOR EMBEDDING.

Still, there’s a problem…

If we take two sentences say “Earth is bigger than Moon“ and “Moon is bigger than Earth“.

The meaning of both is opposite, but their tokens are same as the words are same in both the sentences though their positions are different.

Exactly, the position matters, the way the words are arranged makes sense, so we just can’t vectorize them, we need to take care of their positions too.

Positional encoding

So, the sentences have different meaning due to positions of the words in it.

We know this, but the machine doesn’t, hence we need to specify it by Positional encoding

The part of making sense - is done by positional encoding.

Every token is given a little info related to its context, where now two same words or tokens might not have same position.

You can refer to this article for in depth explanation of Positional encoding.

Phase 2: Understanding the Input

Self attention mechanism

This is simply about - which word or token needs more attention.

For example, let’s take a sentence “He poured water into the glass because it was empty.“

Now the words/tokens here communicate with each other and make relationships among them, so that which one needs attention can be determined.

If we look at the sentence, we as a human know - it was empty, here “it“ refers to the glass.

How could the AI determine it, so the token ‘it‘ goes around and makes connections with words to make sense and give proper context at the end.

Attention is the brain of the model. It lets every word look at other words in the sentence and ask,
“Who matters to me right now?”

This is just one case, there is a lot to understand, so we need multiple forms of attention, hence the “Multi head attention“ concept.

Multi head attention

Of course, there is not only a single aspect that needs attention, but there are lots of other aspects to look around, so every aspect is handled by single ‘head‘ and the final output comes as a rich, multi-dimensional view of the sentence.

A simple example of this situation would be,

It’s like having a group of experts in a room — each focusing on a different part of the problem — and then merging their thoughts.

Phase 3: Learning and Decision making

Everything we’ve seen so far — tokenization, embeddings, positional encoding, attention —
all of it happens inside a powerful architecture called the Transformer.

Inside this Transformer there is an encoder and a decoder, and in between, there’s layer after layer of deep neural networks.

We have seen encoding part, decoding will be done at the end.

Next up is neural networks, something which we already born with…😉

Neural Networks

Neural networks can be understood easily if we think of it as our brains 🧠.

Our brains learn by building associations.
You people remember toppers?
how they used to learn easily and faster than others.
They used to create mnemonics, like:

🎨 Resistor color code?
BB ROY of Great Britain had a Very Good Wife

This is learning by connecting ideas — and the brain reinforces the strongest, smartest ones.

AI tries to do the same thing with neural networks.
Instead of neurons, we’ve here artificial ones. Instead of mnemonics, we’ve here the weights that adjust over time.
When a prediction is right, we reinforce that path. When it’s wrong, we adjust it — until the system “remembers” what works best.

How do we adjust in Ai world

That’s where backpropagation comes in.

Back propagation and Weights

It is like feedback which we humans take to improve and not to repeat any mistake of past.

For example, imagine there is a little kid, who knows nothing about right or wrong.
They don’t know what’s right or wrong, and explore and learn on the go.

So, one day, the kid pushes another child.
The parents scold them. Maybe a slap on the hand. The message is loud and clear that it is not okay…

Next time, the kid helps someone — picks up a fallen toy, shares a snack.
This time, they get good appreciation, maybe chocolate…
The brain of the kid notes it down: “That’s good. Let’s do that again.”

That’s how kids learn. Not by being told everything upfront, but by trial and error, followed by feedback — negative or positive.

Now AI isn’t much different.

When a model makes a prediction — say, the next word in a sentence — it doesn’t know it’s wrong right away.
But once the error is calculated, it sends that info back through the model (backpropagation.,.,.,.,).

Bad guesses = reduce the weights.
Good guesses = increase the weights.

Weight is just a number that determines how strong the connection is,

In neural networks, weights are numerical values associated with connections between neurons, determining the strength of those connections and influencing the network's predictions and learning process.

Just like a child, over time the model learns what “feels right” based on rewards and corrections.

Hope you got it 🙃.

Phase 4: Generating the output

Vocab

Vocab is short for Vocabulary, it is just what the complete set of all tokens the model knows

So when we say "the model has a vocab of 50,000 tokens", it means:

The model can only understand and generate from those 50,000 pieces (words, sub-words, punctuation, etc.)

So, each vocab has 3 things with it

Unique ID

Vector representation

Logit score

Softmax

Now, it’s time for response - output for the input provided by user

From the trnasformer, after iterating few times, the output is generated asa scores - numbers assigned to each word in the “vocab“

We’ll see what is vocab next.

raw scores are like

• "the": 4.2

• "AI": 7.9

• "is": 6.3, ...

So these raw scores of output tokens are converted into probability - such that it sums up to 100%

The one with higher percentage will be the ‘output - next word‘, something like this…

• "AI": 72%

• "is": 20%

• "the": 8%

Temperature

So, think of temperature as the level of randomness in the model’s replies.

This handles the randomness part of the response of an AI model

As we are aware of softmax, it is used to assign probabilities to all tokens in its vocab.

Now comes the temperature part, it tweaks these probabilities before the model picks the next token.

Let’s take a small example

The lesser the temperature, there’s highest probability that model will select this often (here, "dog")

The higher the temperature, flattens the probabilities, giving less likely words (like "spaceship") a better chance of being picked.

Decoder

The decoder picks the next token based on those probabilities (temperature will affect this).

Then it does the same steps again (with the new token added) to generate the next word, and so on…

Until you get the full output…

Conclusion

Perfect, now do you remember the paragraph we read in the beginning?

Your words get tokenized—broken into tiny pieces—then turned into vector embeddings that carry their meaning. These embeddings pass through encoders, layer after layer, guided by attention to focus on the right parts. They travel through a huge transformer model made of deep neural networks, trained on massive amounts of data. The AI looks at everything within its context window, uses learned weights to understand what matters, and runs softmax math to guess the next word. The decoder then builds your reply, picking from a giant vocab.

Well… now you’ve decoded it 😤.

So, if we do a quick recap of what we have learnt so far,

• Tokens are just numberized word-pieces,

• Vector embeddings carry meaning into math,

• Attention helps the model focus,

• Transformers guide the flow,

• Neural networks learn like our brains do,

• Backpropagation corrects mistakes,

• Weights are memory strength,

• Softmax gives options, and temperature adds randomness,

• Decoder picks each next word from a massive vocab.

So, the next time you chat with an AI, you won’t just think it’s magic.

You’ll see the logic.

And you’ll know:

it’s not a mystery.

It’s math—beautiful, layered, structured math 👀.

Don’t forget to drop a like ❤️ and a comment if you loved it / learned something by reading this article, feel free to connect with me here on X

Peace out ✌️