Index file

The Bare Basics

Storing Data on Disks and Files

Chapter 9

Disks and Files
 DBMS stores information on (“hard”) disks.

 This has major implications for DBMS design!

 READ: transfer data from disk to main memory (RAM).
 WRITE: transfer data from RAM to disk.

 Both are high-cost operations, relative to in-memory
operations, so must be planned carefully!

Why Not Store Everything in Main
Memory?
 Costs too much.
 Same amount of money will buy you say either 128MB of RAM or
20GB of disk.

 Main memory is volatile.
 We want data to be saved between runs. (Obviously!)

 Typical storage hierarchies:
 Main memory (RAM) for currently used data (primary storage) .
 Disk for the main database (secondary storage).
 Tapes for archiving older versions of data (tertiary storage).

Disks
 Secondary storage device of choice.
 Main advantage over tapes:
 random access vs. sequential.
 Data is stored and retrieved in units :
 called disk blocks or pages.

 Unlike RAM, time to retrieve a disk page
varies depending upon location on disk.
 Therefore, relative placement of pages on disk has
major impact on DBMS performance!

Components of a Disk Spindle
Tracks
 The platters spin Disk head

(say, 90 rps).
The arm assembly is Sector

moved in or out to
position a head on a
desired track.
Tracks under heads Platters
Arm movement
make a cylinder
(imaginary!).
Only one head
reads/writes at any Arm assembly
one time.
 Block size is a multiple
of sector size (which is fixed).

Accessing a Disk Page
 Time to access (read/write) a disk block:
 seek time (moving arms to position disk head on track)
 rotational delay (waiting for block to rotate under head)
 transfer time (actually moving data to/from disk surface)
 Seek time and rotational delay dominate.
 Seek time varies from about 1 to 20msec
 Rotational delay varies from 0 to 10msec
 Transfer rate is about 1msec per 4KB page

 Lower I/O cost: reduce seek/rotation delays!

Arranging Pages on Disk
 `Next’ block concept:
 blocks on same track, followed by
 blocks on same cylinder, followed by
 blocks on adjacent cylinder

 Blocks in a file should be arranged sequentially on
disk (by `next’), to minimize seek and rotational
delay.

 For a sequential scan, pre-fetching several pages at a
time is a big win!

RAID (Redundant Array of Independent
Disks)
 Disk Array: Arrangement of several disks that gives
abstraction of a single large disk.

 Goals: Increase performance and reliability.

 Two main techniques:
 Data striping:
Data is partitioned;
Size of a partition is called the striping unit.
Partitions are distributed over several disks.
 Redundancy:
More disks => more reliable.
Redundant information allows reconstruction of data if a
disk fails.

RAID Levels

 Level 0: No redundancy
 Best write performance
 Not best in reading. (Why?)

 Level 1: Mirrored (two identical copies)
 Each disk has a mirror image (check disk)
 Parallel reads, a write involves two disks.
 Maximum transfer rate = transfer rate of one disk

RAID Levels
 Level 0+1: Striping and Mirroring
 Parallel reads, striping unit is block
 a write involves two disks.
 Maximum transfer rate = aggregate bandwidth

 Level 2: Error-Correcting Codes
 Striping unit is bit (D datadisks + C checkdisks)
 Redundancy scheme is Hamming code
 Smallest reading unit is D blocks (suitable for large requests)
 Writing to D+C disks
 Effective space utilization increases with the number of data
disks

1

RAID Levels (Contd.)
 Level 3: Bit-Interleaved Parity
 Striping Unit: One bit. One check disk.
 Each read and write request involves all disks;
disk array can process one request at a time.
 Level 4: Block-Interleaved Parity
 Striping Unit: One disk block. One check disk.
 Parallel reads possible for small requests, large requests can
utilize full bandwidth
 Writes involve modified block and check disk
 Level 5: Block-Interleaved Distributed Parity
 Similar to RAID Level 4, but parity blocks are distributed over
all disks
 Advantages?
write? Bottleneck ---- check disk
Read? All disk involve reading (no check disk)
1

Choosing RAID Levels
 RAID Level 0: data loss is not an issue
 RAID Level 0+1:
 small storage subsystems, the cost of mirroring is moderate
 High percentage of writes
 RAID Level 3
 Large transfer requests of several contiguous blocks
 RAID Level 5
 A good general-purpose solution
 Good performance for large as well as small requests

1

Disk Space Management
 Lowest layer of DBMS software manages space
on disk.

 Higher levels call upon this layer to:
 allocate/de-allocate a page
 read/write a page

 Higher levels don’t need to know how this is
done, or how free space is managed.

1

Buffer Management in a DBMS
Page Requests from Higher Levels

BUFFER POOL

disk page

free frame

MAIN MEMORY

DISK choice of frame dictated
DB by replacement policy

 Data must be in RAM for DBMS to operate on it!
 Table of <frame#, pageid> pairs is maintained.
1

When a Page is Requested ...
 If requested page is not in buffer pool:
 Choose a frame for replacement
 If frame is dirty, write it to disk
 Read requested page into chosen frame

 Pin the page and return its address.

If requests can be predicted (e.g., sequential scans)
pages can be pre-fetched (several pages at a time)!

1

More on Buffer Management
 Requestor of page must unpin it, and indicate
whether page has been modified:
 dirty bit is used for this.

 Page in pool may be requested many times,
 a pin count is used.
 A page is a candidate for replacement iff pin count = 0.

 CC & recovery may entail additional I/O when a
frame is chosen for replacement. (Write-Ahead
Log protocol; more later.) 1

Buffer Replacement Policy
 Frame is chosen for replacement by a
replacement policy:
 Least-recently-used (LRU), Clock, MRU etc.
 Policy can have big impact on # of I/O’s;
depends on access pattern.
 Sequential flooding: Nasty situation caused by
LRU + repeated sequential scans.
 # buffer frames < # pages in file means each page
request causes an I/O.
 MRU much better in this situation (but not in all
situations, of course).
1

DBMS vs. OS File System
OS does disk space & buffer mgmt already!
So why not let OS manage these tasks?

 Differences in OS support: Portability issues
 Some limitations, e.g., files don’t span multiple disk devices.
 Buffer management in DBMS requires ability to:
 pin a page in buffer pool,
 force a page to disk (important for implementing CC &
recovery),
 adjust replacement policy, and pre-fetch pages based on access
patterns in typical DB operations.

1

These layers
Structure of a DBMS must consider
concurrency
control and
recovery
 A typical DBMS has a layered
Query Optimization
architecture.
and Execution
 Disk Storage hierarchy, RAID
 Disk Space Management Relational Operators
Roles, Free blocks Files and Access Methods
 Buffer Management
Buffer Pool, Replacement policy Buffer Management
 Files and Access Methods
Disk Space Management
File organization (heap files, sorted file,
indexes)
File and page level storage (collection
Index Files
of pages or records)
Data Files
DBSystem Catalog

1

Files of Records
 Page or block is the granularity for doing I/O
 Higher levels of DBMS operate on :
 records, and
 files composed of records.
 FILE: A collection of pages, each containing a
collection of records.
 File must support:
 insert/delete/modify record
 read a particular record (specified using record id)
 scan all records (possibly with some conditions on
the records to be retrieved)
2

Unordered Files (Heap Files)
 Simplest file structure contains records in no
particular order.

 As file grows and shrinks, disk pages are allocated
and de-allocated.

 To support record level operations, we must:
 keep track of the pages in a file
 keep track of free space on pages
 keep track of the records on a page

 There are many alternatives for keeping track of this.
2

Alternative 1:
Heap File Implemented as List
Data Data Data Full Pages
Page Page Page
Header
Page
Data Data Data
Pages with
Page Page Page
Free Space

 Maintain a table containing pairs of:
<heap_file_name, head_page_address>
 Each page contains 2 `pointers’ (rid) plus data.

2

Heap File Implemented as a List
 Insert a new page into heap file
 Disk manager adds a new free space page into link

 Delete a page from heap file
 Removed from the list
 Disk manager deallocates it

 Disadvantages:
 If records are of variable length, all pages will be in free list.
 Retrieve and examine several pages for enough space.

2

Alternative 2: Heap File Using Page Directory
Data
Header Page 1
Page
Data
Page 2

Data
DIRECTORY Page N

 In directory, each entry for a page includes number of
free bytes on page.
 The directory is a collection of pages
(linked list implementation is just one alternative).
 Much smaller than linked list of all HF pages!
2

Alternative 2:
Heap File Using a Page Directory

 Advantage of Page Directory :
 The size of directory is very small (much smaller
than heap file.)
 Searching space is very efficient, because find free
space without looking at actual heap data pages.

2

Page Formats
 Page : abstraction is used for I/O
 Record : data granularity for higher level of DBMS

 How to arrange records in pages?
 Identify a record:
• <page_id, slot_number>, where slot_number = rid
• Most cases, use <page_id, slot_number> as rid.

 Alternative approaches to manage slots on a page
 How to support insert/deleting/searching?

2

Records Formats: Fixed Length Record
F1 F2 F3 F4

L1 L2 L3 L4

Base address (B) Address = B+L1+L2

 Information about field types same for all records
in a file
 Stored record format in system catalogs.

+ Finding i’th field does not require scan of record,
just offset calculation.
2

Page Formats: Fixed Length Records
Slot 1 Slot 1
Slot 2 Slot 2
Free
... Space ...
Slot N Slot N

Slot M
N 1 . . . 0 1 1M
number M ... 3 2 1 number
PACKED of records UNPACKED, BITMAP of slots

Record id = <page id, slot #>.

Note: In first alternative, moving records for free space
management changes rid; may not be acceptable if
existing external references to the record that is moved.
2

Record Formats: Variable Length
 Two alternative formats (# fields is fixed):
F1 F2 F3 F4

4 $ $ $ $

Field
Fields Delimited by Special Symbols
Count
F1 F2 F3 F4

Array of Field Offsets

+ Second offers direct access to i’th field
+ efficient storage of nulls ;
- small directory overhead. 2

Page Formats: Variable Length Records

Rid = (i,N) Length = 20 Offset of
Page i record
from start
Rid = (i,2) Length = 16 of data
area
Rid = (i,1) Length = 24

20 16 24 N Pointer
N ... 2 1 # slots to start
of free
space
SLOT DIRECTORY

Slot directory = {<record_offset, record_length>}
3

Page Formats: Variable Length Records

Slot directory = {<record_offset, record_length>}
Dis/Advantages:
+ Moving: rid is not changed
+ Deletion: offset = -1 (rid changed?
Can we delete slot? Why?)
+ Insertion: Reuse deleted slot.
Only insert if none available.

Free space? Free space pointer? Recycle after deletion?

3

System Catalogs
 Meta information stored in system catalogs.

 For each index:
 structure (e.g., B+ tree) and search key fields
 For each relation:
 name, file name, file structure (e.g., Heap file)
 attribute name and type, for each attribute
 index name, for each index
 integrity constraints
 For each view:
 view name and definition
 Plus statistics, authorization, buffer pool size, etc.

Catalogs are themselves stored as relations!
3

Attr_Cat(attr_name, rel_name, type, position)
attr_name rel_name type position
attr_name Attribute_Cat string 1
rel_name Attribute_Cat string 2
type Attribute_Cat string 3
position Attribute_Cat integer 4
sid Students string 1
name Students string 2
login Students string 3
age Students integer 4
gpa Students real 5
fid Faculty string 1
fname Faculty string 2
sal Faculty real 3
3

Summary
 Disks provide cheap, non-volatile storage.
 Random access, but cost depends on location of page
on disk
 Important to arrange data sequentially to minimize
seek and rotation delays.
 Buffer manager brings pages into RAM.
 Page stays in RAM until released by requestor.
 Written to disk when frame chosen for replacement.
 Frame to replace based on replacement policy.
 Tries to pre-fetch several pages at a time.

3

More Summary
 DBMS vs. OS File Support
 DBMS needs features not found in many OSs.
• forcing a page to disk
• controlling the order of page writes to disk
• files spanning disks
• ability to control pre-fetching and page replacement
policy based on predictable access patterns
 Formats for Records and Pages :
 Slotted page format : supports variable length
records and allows records to move on page.
 Variable length record format : field offset
directory offers support for direct access to i’th
field and null values.

3

Even More Summary
 File layer keeps track of pages in a file, and
supports abstraction of a collection of records.
 Pages with free space identified using linked list
or directory structure

 Indexes support efficient retrieval of records
based on the values in some fields.

 Catalog relations store information about
relations, indexes and views.
 Information common to all records in collection.
3

Index file

More Related Content

What's hot (20)

Similar to Index file (20)

Recently uploaded (20)

Index file