Chroma Memory implementation in Asp.net Razor Pages

[Imran Sh (imranshams)]

I'm trying to develop a razor page which has Chroma embedded memory me based on this link

Chroma is running and I can search for a question. Logs of my questions:

what is my name? My name is Andrea
where do I live? I currently live in Seattle and have been living there since 2005
where is my family from? My family is from New York
where have I travelled? I visited France and Italy five times since 2015
what do I do for work? I currently work as a tourist operator

From my base class:

protected string GetMemoryCollectionName()
        {
            return _MemoryCollectionName;
        }

protected async Task<ISemanticTextMemory> GetMemoryAsync()
        {

            var memoryBuilder = new MemoryBuilder();

            memoryBuilder.WithAzureOpenAITextEmbeddingGeneration(
                deploymentName: "text-embedding-ada-002",
                endpoint: _azureOpenAIConfig.EndPoint,
                apiKey: _azureOpenAIConfig.ApiKey,
                "azureml://registries/azure-openai/models/text-embedding-ada-002/versions/2");



            var chromaMemoryStore = new ChromaMemoryStore("http://127.0.0.1:8000");

            

            memoryBuilder.WithMemoryStore(chromaMemoryStore);

            var memory = memoryBuilder.Build();

            await memory.SaveInformationAsync(_MemoryCollectionName, id: "info1", text: "My name is Andrea");
            await memory.SaveInformationAsync(_MemoryCollectionName, id: "info2", text: "I currently work as a tourist operator");
            await memory.SaveInformationAsync(_MemoryCollectionName, id: "info3", text: "I currently live in Seattle and have been living there since 2005");
            await memory.SaveInformationAsync(_MemoryCollectionName, id: "info4", text: "I visited France and Italy five times since 2015");
            await memory.SaveInformationAsync(_MemoryCollectionName, id: "info5", text: "My family is from New York");

            return memory;
        }

The issue beging after integration of memory with PromptFunction:

                 var memory = await GetMemoryAsync();

                 var questions = new[]
                 {
                           "what is my name?",
                           "where do I live?",
                           "where is my family from?",
                           "where have I travelled?",
                           "what do I do for work?",
                };

                foreach (var q in questions)
                {
                    var responseX = await memory.SearchAsync(GetMemoryCollectionName(), q, limit: 1, minRelevanceScore: 0.5).FirstOrDefaultAsync();
                    Console.WriteLine(q + " " + responseX?.Metadata.Text);
                }

// TextMemoryPlugin provides the "recall" function
                _kernel.ImportPluginFromObject(new TextMemoryPlugin(memory));

                const string skPrompt = @"
ChatBot can have a conversation with you about any topic.
It can give explicit instructions or say 'I don't know' if it does not have an answer.

Information about me, from previous conversations:
- {{$fact1}} {{recall $fact1}}
- {{$fact2}} {{recall $fact2}}
- {{$fact3}} {{recall $fact3}}
- {{$fact4}} {{recall $fact4}}
- {{$fact5}} {{recall $fact5}}

Chat:
{{$history}}
User: {{$userInput}}
ChatBot: ";

                var chatFunction = _kernel.CreateFunctionFromPrompt(skPrompt, executionSettings);


                var arguments = new KernelArguments();

                arguments["fact1"] = "what is my name?";
                arguments["fact2"] = "where do I live?";
                arguments["fact3"] = "where is my family from?";
                arguments["fact4"] = "where have I travelled?";
                arguments["fact5"] = "what do I do for work?";

                arguments[TextMemoryPlugin.CollectionParam] = GetMemoryCollectionName();
                arguments[TextMemoryPlugin.LimitParam] = "2";
                arguments[TextMemoryPlugin.RelevanceParam] = "0.8";

                var history = "";
                foreach (var item in aiHistory)
                {
                    switch (item.Role.Label.ToLower())
                    {
                        case "assistant":
                            history += "ChatBot: " + item.Items.Last() + "\n";
                            break;

                        case "user":
                            history += "User: " + item.Items.Last() + "\n";
                            break;

                        default:
                            break;
                    }
                }
                arguments["history"] = history;
                arguments["userInput"] = aiHistory.Last().Items.Last();
                var response = await _kernel.InvokeAsync(chatFunction, arguments);

When I ask the following question:

Hello, I think we've met before, remember? my name is...

I received the following answer:

Hello! I'm sorry, but I don't have access to previous conversations, so I don't remember your name. Could you remind me?

That means the LLM has no memory in initial state!

I have no error and exception!
What I did wrong? Is anything missed?
How should I diagnose the problem in same case?

Source code:
Code.zip

[Dmytro Struk (dmytrostruk)]

Hi Imran Sh (@imranshams) !
Did you save the information in Chroma DB before calling kernel.InvokeAsync?

const string MemoryCollectionName = "aboutMe";

await memory.SaveInformationAsync(MemoryCollectionName, id: "info1", text: "My name is Andrea");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info2", text: "I currently work as a tourist operator");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info3", text: "I currently live in Seattle and have been living there since 2005");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info4", text: "I visited France and Italy five times since 2015");
await memory.SaveInformationAsync(MemoryCollectionName, id: "info5", text: "My family is from New York");

[Imran Sh (imranshams)]

Hi Dmytro Struk (@dmytrostruk)
Yes, I updated the question and attached my code as well if you need.

[Dmytro Struk (dmytrostruk)]

From issue description, it looks like you see the results from DB when you try to search directly, but it's not working when executing a function with Kernel. In this line, could you try to decrease relevance score to 0.5?

arguments[TextMemoryPlugin.RelevanceParam] = "0.8"; // set to 0.5

[Imran Sh (imranshams)]

Thats was the issue, it works with 0.7 as well. I take the parameter same as here.

By the way, thank you so much for your support and effort.

[Federico Laurianti (Laurianti)]

I know this is old and answered, but the stack here has changed since: the memory store approach (TextMemoryPlugin over the Chroma memory store) is deprecated, and #14579 removed the .NET Chroma connector. For anyone doing the same today: Semantic Kernel works with the Microsoft.Extensions.VectorData abstractions, and the Chroma implementation is CommunityToolkit.VectorData.Chroma, in the AI Community Toolkit. A sample with the same flow, searching Chroma and passing the results to the model: ChatCompletion_Rag_Chroma.