Welcome to the Fedora Performance and Scalability Wiki

Overall goal

It is of paramount importance for large-scale applications that Fedora Commons can handle huge amounts of data efficiently. While Fedora is generally known to be stable and reliable, there appears to be a lack of data and experience regarding large-scale installations and the performance implications thereof.

There are some known performance issues, namely regarding the ingest of new objects. It remains to be investigated if there can be some room for improvement. A faster ingest would be beneficial for many large projects using Fedora Commons.
The aim of this Wiki therefore is to gather data, document limits and constraints, devise recommendations and help improve Fedora Commons evolve further by contributing the findings to the community.

Latest Findings

We have now successfully loaded 14 million objects (see TestData) with roughly 750 million triples into Fedora. The ingest rate of about 10 objects/second was steady across the whole ingest process, which took roughly 21 days. Please find more details here.

About this Wiki

The Wiki consists of four distinct sections:
  • Understanding Fedora explains certain aspects which are not currently covered by the Fedora documentation, namely functionality deep inside the system which is normally not important for the end user to know but bears significance performance-wise.
  • Measurement documents some aspects about the basis of the measurement approach itself. In order to understand the test runs and the findings better, a context has to be provided.
  • Test Runs documents the executed tests and their outcome. This is the core of the wiki.
  • Conclusions provide the derived recommendations for large-scale Fedora Commons projects. This section contains the key observations and deduced recommendations.

Add new attachment

In order to upload a new attachment to this page, please use the following box to find the file, then click on “Upload”.
« This page (revision-5) was last changed on 17-Jul-2012 15:51 by KST [RSS]