Journaled File LIBrary (libjf) FAQ

"Frequently Asked Questions"

Christian Ferrari

          
        


Table of Contents
1. Global
2. Development
2.1. Testing
3. System
3.1. Utilities
3.2. Recovery
3.3. Tuning
A. GNU Free Documentation License
A.1. PREAMBLE
A.2. APPLICABILITY AND DEFINITIONS
A.3. VERBATIM COPYING
A.4. COPYING IN QUANTITY
A.5. MODIFICATIONS
A.6. COMBINING DOCUMENTS
A.7. COLLECTIONS OF DOCUMENTS
A.8. AGGREGATION WITH INDEPENDENT WORKS
A.9. TRANSLATION
A.10. TERMINATION
A.11. FUTURE REVISIONS OF THIS LICENSE
A.12. ADDENDUM: How to use this License for your documents

1. Global

1.1. Can I help fixing English language related mistakes of these FAQs?
1.2. Can I suggest new FAQs?
1.3. Is libjf "Yet Another Database Tool"?
1.4. Is libjf "Yet Another Journaled File System"?
1.5. Is there a similar tool?
1.6. Is there a "step by step tutorial"?
1.7. Is there an "API reference guide"?
1.8. What did convince you to develop libjf?
1.9. What's a "journal file"?
1.10. What's a "journaled file"?
1.11. What's a "tested environment"?
1.12. Is there any warranty if I use libjf in a tested environment?
1.13. How can I add a new "tested environment" to doc/tested_arch.xml list?
1.14. What's "recovery pending" status for a journal?
1.15. What's "damaged journal" status for a journal?
1.16. Is there a porting for Microsoft Windows operating systems?
1.17. How slow is libjf compared to C standard streams (stdio)?

1.1. Can I help fixing English language related mistakes of these FAQs?

Sure, please send me a better version at

1.2. Can I suggest new FAQs?

Sure, please send me it at

1.3. Is libjf "Yet Another Database Tool"?

No, libjf is a tool can be used to add application and/or system transactionality to your file based applications: it does not matter how your files are used (text, binary, search tree, heap, etc...). An interesting feature is the "transactional read" feature you can use with "R" and "R+" open modes: you can start reading from the point you leaved at previous execution without writing this information in some different place.

1.4. Is libjf "Yet Another Journaled File System"?

No, libjf API offers application level recovery and is independent from specific filesystem/operating system. The effort is to develop only "ANSI C" code, but some exceptions are necessary to avoid bugged or foolish code.

1.5. Is there a similar tool?

Alberto Bertogli noticed me he developed libjio. "Libjf" and "libjio" are distinct original implementations that solve slightly different problems in the same class: please feed-back me your opinion about "libjf vs. libjio".

1.6. Is there a "step by step tutorial"?

Yes, it's in doc/tutorial directory.

1.7. Is there an "API reference guide"?

Yes, it's in doc/api_reference directory.

1.8. What did convince you to develop libjf?

I was interested in developing a search technology to manage impressive number of keys, from 10^12 up, with search time and insert time bounded by O(K) where K is the length of the key you are searching/inserting. After some early drafts I realized there's no usage for a huge index if recovery is not guaranteed: you can not insert again 10^12 keys because an application/system crash happened while updating the index and something "gone wrong". A journaling technology was necessary. Developing and improving libjf is a hard task so I don't think I'll be able to develop "tera index" technology too.

1.9. What's a "journal file"?

It's a special file handled by libjf library and used to store transactional data related to one or more journaled files.

1.10. What's a "journaled file"?

It's a normal file created and managed by libjf: using libjf API you can perform ACID transactions on one or more journaled files as you are used with relational databases. A journaled file can be read with any program like cat, less, etc..., but can be written only using libjf API. Writing a journaled file with different tools breaks recovery properties and the file may be damaged when journal file is opened.

1.11. What's a "tested environment"?

It's a tuple (operating system, libc, C compiler) with these properties:

  • libjf compiles without warning

  • stress_test.sh executes without errors.

1.12. Is there any warranty if I use libjf in a tested environment?

No. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.

1.13. How can I add a new "tested environment" to doc/tested_arch.xml list?

Please be sure the new environment complies with these properties:

  • libjf compiles without warnings

  • stress_test.sh executes without errors.

Fill in a new tested_arch_record.xml record specifying all tags. Supposing you successfully tested the library distributed with package libjf-1.2.3-4.2.1-20051231.tar.gz you may retrieve the information you need using this procedure:

  • package_version: first 3 numbers, 1.2.3

  • library_version: following 3 number, 4.2.1

  • date: execute command

    date +%Y-%m-%d
    	  

  • description: take a look to original operating system package (CDs, DVDs, etc...)

  • config_guess: execute command

    ./config.guess
    	  
    example:
    ./config.guess
    i686-suse-linux
    	  

    • host_cpu: "i686"

    • host_vendor: "suse"

    • host_os: "linux"

  • libc_version: on GNU/Linux rpm-based systems, the command is

    rpm -qf /lib/libc.so.6
    	  
    on different operating systems you should use the specific packaging tool ("smit", "sam", "*pkg*", ...)

  • cc_version: many C compiler accepts -v option to retrieve the version:

    cc -v
    	  
    some C compilers may need something else...

Records lacking of any data will not be included in official documentation. Send to your fresh tested_arch_record.xml

1.14. What's "recovery pending" status for a journal?

"Recovery pending" means: "application and/or system crashed before a valid synchronization point and some operations must be performed in order to recover journaled files". "Recovery pending" is a common status in a transactional system.

1.15. What's "damaged journal" status for a journal?

"Damaged journal" means: "the journal contains at least a record that's not well formed". Recovery can be performed only truncating the journal at last valid record. If damage appears in the middle of the journal - not at tail position - the journal itself is compromised and recovery is not suggested. "Damaged journal" must not be common status in a transactional system.

1.16. Is there a porting for Microsoft Windows operating systems?

A native porting is not yet available, but cygwin emulation has been successfully used to compile and test libjf. A native porting will be done, but there's no a release date for this milestone.

1.17. How slow is libjf compared to C standard streams (stdio)?

libjf has to make a lot of work to supply transactional integrity so the comparison is not a 1:1 race. Take a look to Section 3.3 to read more about libjf benchmarking.