Remote Database vs Local Database

Test Setup

  • Java 1.6, no optimizations
  • Tomcat 5.5 (Fedora default)
  • Triplestore MPT
  • 1GBit network connection to remote database host
  • Postgresql 8.3 on remote host, default installation
  • 50000 digital objects with external managed contend supplied locally via tomcat context

Local Database

# objectstotal (ms)avg (ms)min (ms)max (ms)
50,0006,623,684132453,990

Remote Database

# objectstotal (ms)avg (ms)min (ms)max (ms)
50,0004,900,75498451,957

The captured data reveal that ingesting objects using a remote database is a much faster approach (in this case about 28% total). The images below may help understand the outcome better:

A simple moving average of both measurements should help identify the underlying trends better:

Conclusion

Using a remote database for the ingest is clearly faster. The benefit originates probably from the hard disk not having to write the ingested object as well as persisting the database all the time. The head of the disk does therefore not have to jump incessantly to locations physically far apart which in turn saves time.
The outcome suggests that pursuing the approach of distributing io is promising. Other tests have to be done involving:
  • What is the impact of getting managed content from a remote host ?
  • What is the impact of getting objects to be ingested from a remote host ?
  • How can io be minimized in Fedora ?

Add new attachment

In order to upload a new attachment to this page, please use the following box to find the file, then click on “Upload”.

List of attachments

Kind Attachment Name Size Version Date Modified Author Change note
png
fedrep1_managed_remotevslocald... 83.701 kB 1 Mon May 26 17:45:44 CEST 2008 KST
png
fedrep1_managed_remotevslocald... 17.826 kB 1 Mon May 26 17:45:57 CEST 2008 KST
png
fedrep1_managed_remotevslocald... 22.832 kB 1 Mon May 26 17:46:07 CEST 2008 KST
png
fedrep1_managed_remotevslocald... 9.934 kB 1 Mon May 26 17:46:21 CEST 2008 KST
« This page (revision-1) was last changed on 21-May-2008 12:21 by unknown [RSS]