LongMem is Microsoft’s answer for unlimited context length

-
  • The framework makes it possible to have unlimited context length.
  • An unlimited context length means a more personalized conversation with AI models.
  • It's basically the next step for AI in fully becoming near human.microsoft longmem

  • Is LongMem the Microsoft answer for unlimited context length?

    Well, to understand what it means to have an unlimited context length, we first need to understand what does context length mean?

    Context length refers to the number of tokens (words, signs, etc) permitted by a model, its input and output, and yours as well.microsoft longmem

  • For example, ChatGPT has a limited number of tokens which means its context length is limited as well. As soon as ChatGPT passes that limit, then all the continuous interaction that you had with it up to that point, will lose all significance. Or better said, it will reset.

    So if you’re starting a conversation with ChatGPT on the subject of Windows, and that conversation lasts more than the limited context length, then the AI tool will lose the context, and will either start to deviate from the subject, or the whole conversation will reset.

    An unlimited context length will make sure that won’t happen, and the AI model will continue to answer you on the subject, while also learning and adapting information as you talk to it about Windows.

    That means the AI model will also personalize the conversation according to your input, hence the need for the context to have an unlimited length.

    How does LongMem work?

    Microsoft promises just that with their new research on the framework, LongMem. LongMem would enable large language models to memorize long-term contexts and utilize long-term memory at the reduced CPU power.

    The framework consists of a frozen large language model as the memory encoder, a residual side network as the memory retriever and reader, and a cached memory bank that stores key-value pairs from past contexts.

    In the research done by Microsoft, experiments show that LongMem outperforms baselines on long-text language modeling, long-context understanding, and memory-augmented in-context learning tasks. Plus, long-term memory allows it to utilize more demonstration examples for better learning.

    And the good news is that LongMem will be open-source. So you will be able to study it and learn how to implement the framework in your own AI model. You can check its GitHub profile here.

Comments

Post a Comment

Popular posts from this blog

What is Game?

-
Image
/ The term "game" can refer to various concepts depending on the context: 1. **A Recreational Activity**: In its most general sense, a game is an activity or form of play governed by a set of rules or customs, often undertaken for enjoyment and sometimes used as an educational tool. 2. **Video Games**: In the context of technology and entertainment, a game often refers to a digital game played on electronic devices such as computers, consoles, or mobile devices. These games can range from simple puzzle games to complex, immersive virtual worlds. 3. **Sports**: Games can also refer to organized athletic competitions governed by rules and regulations, such as football, basketball, or chess. 4. **Board Games and Card Games**: These are physical games played on a tabletop using pieces, cards, or other components, often involving strategic thinking, luck, or both. 5. **Role-Playing Games (RPGs)**: These are games where players assume the roles of characters in a fictional setting
Read more

FIX: User profile or private key are not accessible

-
Image
To fix various PC problems, we recommend this tool: This software will repair common computer errors, protect you from file loss, malware, hardware failure and optimize your PC for maximum performance. Fix PC issues and remove viruses now in 3 easy steps: Download this PC Repair Tool  rated  Great  on TrustPilot.com. Click  Start Scan  to find Windows issues that could be causing PC problems. Click  Repair All  to fix issues with Patented Technologies (Exclusive Discount for our readers). A Secure Sockets Layer, also known as an SSL,  is a standard security technology for establishing an encrypted link between a server and a client. Manipulating isn’t all that easy, and it can usually result in error messages if not done right. One good example is when you try to import a Secure Sockets Layer (SSL) private key certificate file into the local computer personal certificate store. Many users  reported  having issues when trying to do so, and have reported receiv
Read more

23AndMe Hacker Leaks New Tranche of Stolen Data

-
Image
  Two weeks after the first data leak from the DNA ancestry service, the threat actor produces an additional 4 million user records they purportedly stole. A threat actor who claimed responsibility for the compromise of the 23AndMe site earlier this month has released a new dataset, including the records of more than 4 million people's genetic ancestry. The cybercriminal, known by the handle Golem, alleges in a cybercrime Dark Web forum the stolen data includes information on, "the wealthiest people living in the US and Western Europe," according to reports. 23andMe spokesperson Andy Kill said in a statement the organization is still trying to confirm whether the most recently leaked data is genuine. Prior to this most recent leak, an Oct. 1 post on a Dark Web forum by Golem claimed they have a total of 20 million individual pieces of 23andMe data and leaked 1 million lines of data as a teaser, along with an offer to bulk sell data profiles. 23andMe confirmed in early Oct
Read more