1.) automatically defining SoA/AoS transformations for data types in the language, and
2.) automatic profiling of code execution counts & array lengths to determine when inserting such a transformation would save more time than the transformation costs.
3.) native SIMD primitive types
4.) GPU backend a la Futhark
5.) native syntax for parallel execution, where the compiler enforces no data sharing (or read-only shared data) between executions.
6.) [more speculative] pluggable backend for this syntax so that you could run these concurrent executions sequentially, using SIMD instructions on the same thread, in multiple threads, on the GPU, or spread across multiple machines in a cluster without altering the code. That'd let you scale up from a few K to a few T without altering your code.
7.) I/O mechanisms that default to mmap, 0-copy vectored I/O, or RDMA as appropriate.
8.) An unfold primitive in either the language or standard library that lets you compute multiple values with a single traversal over a list, ideally while still keeping each individual computation isolated and modular. I find this is a really common pattern in my Kotlin code, and the biggest culprit for still writing manual accumulator loops.
9.) Support for multi-pass initialization in data structures, where you can hold some fields as uninitialized for a later computation, but mark the whole structure as immutable once every field has been computed.
10.) A standard library that uses these parallelization techniques under the hood so you get really good speed for common operations but it just works. For example, Boyer-Moore can be done on the GPU. Many common regexps can be reduced to string search on a common left prefix + a DFA on potential matches, where the prefix search can be done with Boyer-Moore. That opens the possibility of blazingly fast search over gigabytes of data for common regexps like /<a href=["']?([^"'\s])["']?>(.+?)<\/a>/. Many times the algorithms you'd use when your haystack is ~1G are very different from the algorithms you'd use when your haystack is ~1K, which goes back to the utility of an integrated profiler.
It occurs to me that a lot of these boil down to the inefficiency of doing lots of traversals of arrays in a world where touching memory is usually the critical path. So for example, this is slow but modular:
val high = oneBillionFloats.max()
val low = oneBillionFloats.min()
val total = oneBillionFloats.sum()
val average = oneBillionFloats.sum() / oneBillionFloats.size()
var high = Float.MIN_VALUE
var low = Float.MAX_VALUE
var total = 0
var count = 0
oneBillionFloats.forEach {
high = maxOf(high, it)
low = minOf(low, it)
total += it
++count
}
val average = total / count