Feeds:
Posts
Comments

Archive for the ‘san storage’ Category

We are using QIO to store oracle datafiles, the way to access the data is

1: from SGA

2: from SAN cache

3: from underlying disk

To know how short the latency for 2 is (the time between issuing a read PIO and the data return from san cache without access the underling disk), I force oracle to do a full table scan but with db_file_multiblock_read_count setting to 1. The 10046 raw trace file is as following (I trimmed something like “obj#=324855 tim=4046251273329” to save space).

We can see the latency on average is about 250us=0.25ms, that’s not bad, but not as good as I expected.

WAIT #1: nam='db file sequential read' ela= 253 file#=19 block#=934517 blocks=1
WAIT #1: nam='db file sequential read' ela= 258 file#=19 block#=934518 blocks=1
WAIT #1: nam='db file sequential read' ela= 252 file#=19 block#=934519 blocks=1
WAIT #1: nam='db file sequential read' ela= 253 file#=19 block#=934520 blocks=1
WAIT #1: nam='db file sequential read' ela= 1512 file#=19 block#=934521 blocks=1
WAIT #1: nam='db file sequential read' ela= 265 file#=19 block#=934522 blocks=1
WAIT #1: nam='db file sequential read' ela= 261 file#=19 block#=934523 blocks=1
WAIT #1: nam='db file sequential read' ela= 256 file#=19 block#=934524 blocks=1
WAIT #1: nam='db file sequential read' ela= 264 file#=19 block#=934525 blocks=1
WAIT #1: nam='db file sequential read' ela= 254 file#=19 block#=934526 blocks=1
WAIT #1: nam='db file sequential read' ela= 255 file#=19 block#=934527 blocks=1
WAIT #1: nam='db file sequential read' ela= 253 file#=19 block#=934528 blocks=1
WAIT #1: nam='db file sequential read' ela= 255 file#=19 block#=934529 blocks=1

Could the elapse time get even shorter? Yes. For the read in sequential above, our SAN will pre-fetch  blocks after read the first several blocks (depending on the SAN algorithm, it might pre-fetch right after read the first block, or will only do pre-fetch after SAN controller finds that the fetched blocks are in sequential), although pre-fetch is faster, but it still needs to take time.

To rule out the latency brought by pre-fetch (pre-fetch reduces the latency, but doesn’t avoid latency), I tested a sql to order by tons of data which needs temporary tablespace. After the temporary data stored in san cache, it is less likely to be aged out as the duration of the sql is not that long. Check again the raw trace file, this time we can see the elapse time is only about 10us, which means the latency is about 10 us, that’s pretty good value.


WAIT #25: nam='direct path read temp' ela= 9 file number=2002 first dba=464112 block cnt=25
WAIT #25: nam='direct path read temp' ela= 15 file number=2002 first dba=464137 block cnt=1
WAIT #25: nam='direct path read temp' ela= 9 file number=2002 first dba=464138 block cnt=1
WAIT #25: nam='direct path read temp' ela= 8 file number=2002 first dba=464139 block cnt=1
WAIT #25: nam='direct path read temp' ela= 9 file number=2002 first dba=464140 block cnt=1
WAIT #25: nam='direct path read temp' ela= 9 file number=2002 first dba=464141 block cnt=1
WAIT #25: nam='direct path read temp' ela= 8 file number=2002 first dba=464142 block cnt=1
WAIT #25: nam='direct path read temp' ela= 8 file number=2002 first dba=464143 block cnt=1
WAIT #25: nam='direct path read temp' ela= 8 file number=2002 first dba=464144 block cnt=1
WAIT #25: nam='direct path read temp' ela= 8 file number=2002 first dba=464145 block cnt=1
WAIT #25: nam='direct path read temp' ela= 9 file number=2002 first dba=464146 block cnt=1

Read Full Post »