Counting the number of tokens for LLM in the Linux kernel sources and beyond…
  • AI
  • A

Counting the number of tokens for LLM in the Linux kernel sources and beyond…

  • 12:23
  • 16.03.2025
  • MainEditor0
  • 41

This article about the new extension of the transformer architecture – Titan from Google – that allows expanding the limits of LLM to 2 million tokens, prompted me to inquire how many tokens suitable for LLM are contained in the sources of colossal software.

Read also:
Comments

    Relevant news on the topic "AI"

    Also read