[HADOOP-3514] Reduce seeks during shuffle, by inline crcs - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 0.18.0
Fix Version/s: 0.19.0
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed

Description

The number of seeks can be reduced by half in the iFile if we move the crc into the iFile rather than having a separate file.

Attachments

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

hadoop-3514.patch
28/Jul/08 12:12
31 kB
Jothi Padmanabhan
hadoop-3514-v1.patch
18/Jul/08 17:05
26 kB
Jothi Padmanabhan
hadoop-3514-v10.patch
25/Aug/08 17:19
29 kB
Jothi Padmanabhan
hadoop-3514-v11.patch
28/Aug/08 05:00
29 kB
Jothi Padmanabhan
hadoop-3514-v12.patch
08/Sep/08 10:50
30 kB
Jothi Padmanabhan
hadoop-3514-v13.patch
09/Sep/08 07:54
30 kB
Jothi Padmanabhan
hadoop-3514-v2.patch
01/Aug/08 12:09
31 kB
Jothi Padmanabhan
hadoop-3514-v3.patch
06/Aug/08 08:59
29 kB
Jothi Padmanabhan
hadoop-3514-v4.patch
08/Aug/08 14:52
40 kB
Jothi Padmanabhan
hadoop-3514-v5.patch
12/Aug/08 10:46
32 kB
Jothi Padmanabhan
hadoop-3514-v6.patch
13/Aug/08 11:52
28 kB
Jothi Padmanabhan
hadoop-3514-v7.patch
14/Aug/08 06:08
29 kB
Jothi Padmanabhan
hadoop-3514-v8.patch
18/Aug/08 04:22
28 kB
Jothi Padmanabhan
hadoop-3514-v9.patch
21/Aug/08 11:57
28 kB
Jothi Padmanabhan

Issue Links

is related to

HADOOP-3638 Cache the iFile index files in memory to reduce seeks during map output serving

Closed

relates to

HADOOP-4396 sort on 400 nodes is now slower than in 18

Closed

Activity

People

Assignee:: Jothi Padmanabhan

Reporter:: Devaraj Das

Votes:: 0 Vote for this issue

Watchers:: 5 Start watching this issue

Dates

Created:: 08/Jun/08 14:09

Updated:: 08/Jul/09 16:52

Resolved:: 09/Sep/08 13:11