Turn 10 of a 50K-token chat waits 5 seconds, not 49: the prefix cache on RTX Spark, and whether the code compiles

The 146-second wait in my first RTX Spark post was the cost of a brand-new 100K-token prompt. A...

Read Original

Related