USP-V Virtualizing SSD

[Brian Walter]

While i realize that virtualizing and SSD products is not currently supported, has anyone tried?

I am considering virtualizing a pair of texas memory systems ramsan 500 systems. This is for an extremely busy oltp system pushing over 100K iops. i would like to be able to move my redo logs hot (using tiered storage manager) without taking an outage of my database.

I realize HDS has a product to pin luns into cache...but frankly, i need all the cache i can get.

Thoughts?


Brian

[mackem]

I assume that your LUNs are currently the USPV and you are talking about migrating volumes from your USPV on to the faster TMS box??

Last I checked, the max IOPs supported through external ports on the USP was 2,000 and upped to 4,000 on the USPV. That would require a lot of external ports to support the kind of throughput you are going to be talking about and I think the max is 8...... While it is a really good product, performance through UVM is not great by any stretch.

My two pennies worth

[stephen2615]

Wow, what can I say about 100k IOP's? Thats either a pretty fine system or a system that could be fine tuned.

But seriously, external systems are there to do a specific job and that does not include performance. My experiences have not been awe inspiring to say the least and cache issues can be a problem.

Depending on what you are using, there are better alternatives that HTSM and as you did not offer any indication of hardware and software, it makes it hard to offer advice.

All the really big systems that I have been involved with generally don't have problems with redo/transaction logs as they are wonderful sequential write devices. Random reads are likely to be the killer and the RAMSAN is probably not big enough or cheap enough to be of much use for a huge database. I once built a massive database system where no amount of cache made much difference. There is only so much prefetching that a very large system can do when its mostly all random reads.

Cache residency has its moments depending on what your system is doing and I would consider investigating it.

Anyway, this is all speculative as no one would try to virtualise a system without HDS's blessing. Why put your USP/USP V at risk to try something not supported.

I am sure that something else can be done to make your system run better. Eg, if you have Veritas, mirror the volumes to the RAMSAN. That most definitely works with little performance hits for the benefit. If you are running Non Stop SQL on a Non Stop system, then what can I say?

Stephen

[Brian Walter]

Very large Oracle OLTP system running on a fully stuffed Sun M9000. DAS connected with 32 Emulex hba's / 32 USP-V ports. All storage in 4+4 groups internal to the USP-V. Host side striping across around 600 spindles.

I have seen a lot more than 4k per port...not sure really where this number came from. We do lots and lots of very small transactions.

This is an Oracle ASM system, and I can migrate the data hot using ASM, I was just looking for additional options to consider. This is actually mostly for Oracle indexes, redo logs are kind of a side thought, we have found a couple other ways to address them in the DB with some tuning.

Stephan, you are right, nobody would do something unsupported... but exploring all possibilities is what makes us good engineers. We live on the bleeding edge, and they don't call it that for no reason. :)

[stephen2615]

Very large Oracle OLTP system running on a fully stuffed Sun M9000. DAS connected with 32 Emulex hba's / 32 USP-V ports. All storage in 4+4 groups internal to the USP-V. Host side striping across around 600 spindles.

I have seen a lot more than 4k per port...not sure really where this number came from. We do lots and lots of very small transactions.

This is an Oracle ASM system, and I can migrate the data hot using ASM, I was just looking for additional options to consider. This is actually mostly for Oracle indexes, redo logs are kind of a side thought, we have found a couple other ways to address them in the DB with some tuning.

Stephan, you are right, nobody would do something unsupported... but exploring all possibilities is what makes us good engineers. We live on the bleeding edge, and they don't call it that for no reason. :)

Brian,

I have done something very very similar. I had 192 filesystems each of about 450 GB just for data alone on a fully maxed Sunfire 6900. I had 32 ports in fabric mode to a Sun 9990 (USP with four BED's). Every query used every disk. All the data was RAID5 7D+1P. Logs were 2D+2D. I mucked about with Veritas and striping and ended up with a good mix. I originally started off with about 5 TB so we had something to work with as far as performance went.

Almost every I/O was done in 1 MB chunks. The file system stripe size was just under 4 MB as that was how it was designed using the USP's block size, two array groups and 4 BEDs to make each group of LUN's which was carved up to produce each filesystem. This setup meant 2000 IOP's per port was almost impossible. Massive queries often pushed 150 MB's plus per port. Mind you this was with the USP not the V which has double the bandwidth. But this system was only used by a handfull of people at any one time. It just did some wierd things.

I did use virtualisation to provide space for RMAN backups but that also caused WP issues.. I ran out of space inside the box for disk. Backups ran 7 x 24 which was a horrific side affect for such a big system. COW was a wierd insurance policy..

I would dearly love to see how well it could run on using concatentated parity groups as I find this very quick indeed. I have seen outstanding performance gains over the normal array groups. This design was expected to last up to about 500 TB in size but I left...



Stephen