Large scale data sets for "SOE" and "SH"

I’ve added some new pages describing how to build large scale “SOE” and “SH” schemas. I’ve tested them both to 500GB in size and will create larger scale versions as soon as I can borrow hardware to test them at multi terabyte levels. If you’re interested in giving it ago let me know via the comments page and I can try and assist. You can find the the instructions on how to do create 100GB+ schemas for “SOE” here and the how to create 100GB+ “SH” schemas here

Update : I noticed that the scripts for SH use compression by default. This can slow down a load. and make the expected upload much smaller. I’ll upload a new build shortly that fixes this and afew other issues.
blog comments powered by Disqus