Re:Performance Improvement with the StringBuilder for C++
by mjcohen on Friday September 06, 2013 @11:48AM
(#6341002)
I had a similar problem with a gawk (yes, gawk!) program I was writing. I had to accumulate 10,000,000 32-character strings to produce a 320,000,000 (three hundred and twenty million!) character string.
It was taking forever.
I eventually realized that this string reallocation that was being done 10,000,000 times was the problem.
To solve this, I did a two-level accumulation (perhaps three levels would have been better, but two was enough). I first accumulated 3,000 of the 32-character strings (3,000 because that was about the square root of 10,000,000).
I then accumulated the (about) 3,000 of these (about) 100,000 character strings.
The result took about 30 seconds, which was good enough for what I needed to do.
It was taking forever.
I eventually realized that this string reallocation that was being done 10,000,000 times was the problem.
To solve this, I did a two-level accumulation (perhaps three levels would have been better, but two was enough). I first accumulated 3,000 of the 32-character strings (3,000 because that was about the square root of 10,000,000).
I then accumulated the (about) 3,000 of these (about) 100,000 character strings.
The result took about 30 seconds, which was good enough for what I needed to do.