checksum.c File Reference

CRC, manber functions. More...

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <apr_md5.h>
#include <sys/mman.h>
#include "checksum.h"
#include "helper.h"
#include "global.h"
#include "est_ops.h"
#include "waa.h"

Include dependency graph for checksum.c:

Go to the source code of this file.

Data Structures

struct  t_manber_parms
 CRC table. More...
struct  t_manber_data
 Everything needed to calculate manber-hashes out of a stream. More...

Defines

#define MAPSIZE   (256*1024*1024)
#define MANBER_LINELEN   (APR_MD5_DIGESTSIZE*2+1 + 8+1 + 10+1 +10+1 + 1)
 The maximum line length in md5s :
  • MD5 as hex (constant-length),
  • state as hex (constant-length),
  • offset of block,
  • length of block,
  • \n,
  • \0.


Functions

int cs___manber_data_init (struct t_manber_data *mbd, struct estat *sts)
 Initializes a Manber-data structure from a struct estat.
int cs___end_of_block (const unsigned char *data, int maxlen, int *eob, struct t_manber_data *mb_f)
 Returns the position of the last byte of a manber-block.
static int cs__hex2val (char ch)
 Hex-character to ascii.
int cs__two_ch2bin (char *stg)
 -.
int cs__char2md5 (const char *input, char **eos, md5_digest_t md5)
 -.
char * cs__md5tohex (const md5_digest_t md5, char *dest)
 -.
char * cs__md5tohex_buffered (const md5_digest_t md5)
 -.
int cs___finish_manber (struct t_manber_data *mb_f)
 Finish manber calculations.
int cs__compare_file (struct estat *sts, char *fullpath, int *result)
 -.
int cs__set_file_committed (struct estat *sts)
 -.
void cs___manber_init (struct t_manber_parms *mb_d)
int cs___update_manber (struct t_manber_data *mb_f, const unsigned char *data, apr_size_t len)
svn_error_t * cs___mnbs_close (void *baton)
svn_error_t * cs___mnbs_read (void *baton, char *data, apr_size_t *len)
svn_error_t * cs___mnbs_write (void *baton, const char *data, apr_size_t *len)
int cs__new_manber_filter (struct estat *sts, svn_stream_t *stream_input, svn_stream_t **filter_stream, apr_pool_t *pool)
 -.
int cs__read_manber_hashes (struct estat *sts, struct cs__manber_hashes *data)
 -.

Variables

struct t_manber_parms manber_parms
 The precalculated CRC-table.
static struct t_manber_data cs___manber
 The Manber-structure.
const char cs___mb_wr_format [] = "%s %08x %10llu %10llu\n"
 The write format string for md5s.
const char cs___mb_rd_format [] = "%*s%n %x %llu %llu\n"
 The read format string for md5s.


Detailed Description

CRC, manber functions.

Definition in file checksum.c.


Define Documentation

#define MANBER_LINELEN   (APR_MD5_DIGESTSIZE*2+1 + 8+1 + 10+1 +10+1 + 1)

The maximum line length in md5s :

  • MD5 as hex (constant-length),
  • state as hex (constant-length),
  • offset of block,
  • length of block,
  • \n,
  • \0.

Definition at line 104 of file checksum.c.

Referenced by cs___update_manber(), and cs__read_manber_hashes().

#define MAPSIZE   (256*1024*1024)

Definition at line 26 of file checksum.c.

Referenced by cs__compare_file().


Function Documentation

int cs___end_of_block ( const unsigned char *  data,
int  maxlen,
int *  eob,
struct t_manber_data mb_f 
)

int cs___finish_manber ( struct t_manber_data mb_f  ) 

Finish manber calculations.

Calculates the full-file MD5 hash, and copies it into the associated struct estat . - see comment at cs__new_manber_filter() .

Definition at line 268 of file checksum.c.

References t_manber_data::full_md5, t_manber_data::full_md5_ctx, estat::md5, STOPIF, and t_manber_data::sts.

Referenced by cs___mnbs_close(), and cs__compare_file().

Here is the caller graph for this function:

int cs___manber_data_init ( struct t_manber_data mbd,
struct estat sts 
)

Initializes a Manber-data structure from a struct estat.

Definition at line 522 of file checksum.c.

References BUG_ON, cs___end_of_block(), t_manber_data::fpos, t_manber_data::full_md5_ctx, t_manber_data::last_fpos, t_manber_data::manber_fd, STOPIF, and t_manber_data::sts.

Referenced by cs__compare_file(), and cs__new_manber_filter().

Here is the call graph for this function:

Here is the caller graph for this function:

void cs___manber_init ( struct t_manber_parms mb_d  ) 

Definition at line 541 of file checksum.c.

References AC_CV_C_UINT32_T, CS__MANBER_BACKTRACK, CS__MANBER_MODULUS, CS__MANBER_PRIME, and t_manber_parms::values.

Referenced by cs___end_of_block().

Here is the caller graph for this function:

svn_error_t * cs___mnbs_close ( void *  baton  ) 

Definition at line 816 of file checksum.c.

References cs___finish_manber(), CS__MIN_FILE_SIZE, t_manber_data::fpos, t_manber_data::input, t_manber_data::manber_fd, RETURN_SVNERR, STOPIF, STOPIF_SVNERR, and waa__close().

Referenced by cs___mnbs_read(), cs___mnbs_write(), and cs__new_manber_filter().

Here is the call graph for this function:

Here is the caller graph for this function:

svn_error_t* cs___mnbs_read ( void *  baton,
char *  data,
apr_size_t *  len 
)

Definition at line 773 of file checksum.c.

References cs___mnbs_close(), cs___update_manber(), t_manber_data::input, RETURN_SVNERR, STOPIF, and STOPIF_SVNERR.

Referenced by cs__new_manber_filter().

Here is the call graph for this function:

Here is the caller graph for this function:

svn_error_t* cs___mnbs_write ( void *  baton,
const char *  data,
apr_size_t *  len 
)

Definition at line 793 of file checksum.c.

References cs___mnbs_close(), cs___update_manber(), t_manber_data::input, RETURN_SVNERR, STOPIF, and STOPIF_SVNERR.

Referenced by cs__new_manber_filter().

Here is the call graph for this function:

Here is the caller graph for this function:

int cs___update_manber ( struct t_manber_data mb_f,
const unsigned char *  data,
apr_size_t  len 
)

int cs__char2md5 ( const char *  input,
char **  eos,
md5_digest_t  md5 
)

-.

Converts an ASCII string to an MD5 digest.

Exactly the right number of characters must be present.

Definition at line 173 of file checksum.c.

References cs__hex2val(), and STOPIF_CODE_ERR.

Referenced by cb___close_file(), cs__read_manber_hashes(), ops__load_1entry(), and up__parse_prop().

Here is the call graph for this function:

Here is the caller graph for this function:

int cs__compare_file ( struct estat sts,
char *  fullpath,
int *  result 
)

-.

Checks whether a file has changed.

Parameters:
sts Which entry to check
fullpath The path to the file (optionally, else NULL). If the file has been checked already and fullpath is NULL, a debug message can write (null), as then even the name calculation is skipped.
result is set to 0 for identical to old and >0 for changed. As a special case this function returns <0 for don't know if the file is unreadable due to a EACCESS.
Note:
Performance optimization In normal circumstances not the whole file has to be read to get the result. On update a checksum is written for each manber-block of about 128k (but see CS__APPROX_BLOCKSIZE_BITS); as soon as one is seen as changed the verification is stopped.

Definition at line 303 of file checksum.c.

References t_manber_data::block_md5, CF_CHANGED, CF_NOTCHANGED, CF_UNKNOWN, estat::change_flag, cs__manber_hashes::count, cs___end_of_block(), cs___finish_manber(), cs___manber_data_init(), cs__md5tohex_buffered(), CS__MIN_FILE_SIZE, cs__read_manber_hashes(), DEBUGP, cs__manber_hashes::end, t_manber_data::fpos, cs__manber_hashes::hash, hlp__lstat(), t_manber_data::last_state, MAPSIZE, cs__manber_hashes::md5, estat::md5, sstat_t::mode, ops__build_path(), ops__link_to_string(), sstat_t::size, estat::st, STOPIF, and STOPIF_CODE_ERR.

Referenced by cm__find_file_source(), and ops__update_single_entry().

Here is the call graph for this function:

Here is the caller graph for this function:

static int cs__hex2val ( char  ch  )  [inline, static]

Hex-character to ascii.

Faster than sscanf(). Returns -1 on error.

Definition at line 119 of file checksum.c.

Referenced by cs__char2md5(), and cs__two_ch2bin().

Here is the caller graph for this function:

char* cs__md5tohex ( const md5_digest_t  md5,
char *  dest 
)

-.

Puts the hex string of md5 into dest, and returns dest.

Definition at line 207 of file checksum.c.

Referenced by cs__md5tohex_buffered(), and waa__get_waa_directory().

Here is the caller graph for this function:

char* cs__md5tohex_buffered ( const md5_digest_t  md5  ) 

-.

Converts an MD5 digest to an ASCII string in a self-managed buffer.

This function alternates between a few static buffers, to allow for a printf("%s %s\n", cs__md5tohex_buffered(), cs__md5tohex_buffered());

It doesn't use the string caches, because the exact length of the buffers is known ... so the alloc/realloc/strcpy overhead is unnecessary.

Definition at line 250 of file checksum.c.

References cs__md5tohex().

Referenced by ci__nondir(), cm___md5_datum(), cs___end_of_block(), cs___update_manber(), cs__compare_file(), DEBUGP_dump_estat(), hlp___encode_close(), ops__save_1entry(), st__print_entry_info(), up__close_file(), and up__parse_prop().

Here is the call graph for this function:

Here is the caller graph for this function:

int cs__new_manber_filter ( struct estat sts,
svn_stream_t *  stream_input,
svn_stream_t **  filter_stream,
apr_pool_t *  pool 
)

-.

Creates a svn_stream_t pipe, which writes the checksums of the manber hash blocks to the md5s file.

On commit and update we run the stream through a filter, to create the manber-hash data (see md5s) on the fly.

Note:
We currently give the caller no chance to say whether he wants the full MD5 or not. If we ever need to let him decide, he must either
  • save the old MD5
  • or (better!) says where the MD5 should be stored - this pointer would replace mb_f->full_md5 .

Definition at line 860 of file checksum.c.

References cs___manber, cs___manber_data_init(), cs___mnbs_close(), cs___mnbs_read(), cs___mnbs_write(), DEBUGP, filename, t_manber_data::input, ops__build_path(), STOPIF, and STOPIF_ENOMEM.

Referenced by ci__nondir(), rev__get_text_to_stream(), and up__apply_textdelta().

Here is the call graph for this function:

Here is the caller graph for this function:

int cs__read_manber_hashes ( struct estat sts,
struct cs__manber_hashes data 
)

-.

Reads the md5s file into memory.

Parameters:
sts The entry whose md5-data to load
data An allocated struct cs__manber_hashes; its arrays get allocated, and, on error, deallocated. If no error code is returned, freeing of the arrays has to be done by the caller.

Definition at line 985 of file checksum.c.

References AC_CV_C_UINT32_T, BUG_ON, cs__manber_hashes::count, cs___mb_rd_format, cs__char2md5(), DEBUGP, cs__manber_hashes::end, filename, cs__manber_hashes::hash, hlp__calloc(), hlp__realloc(), IF_FREE, MANBER_LINELEN, cs__manber_hashes::md5, ops__build_path(), STOPIF, STOPIF_CODE_ERR, WAA__FILE_MD5s_EXT, waa__open_byext(), and WAA__READ.

Referenced by cs__compare_file().

Here is the call graph for this function:

Here is the caller graph for this function:

int cs__set_file_committed ( struct estat sts  ) 

-.

Callback for the checksum layer.

If a file has been committed, this is where various checksum-related uninitializations can happen.

Definition at line 500 of file checksum.c.

References estat::flags, sstat_t::mode, estat::repos_rev, RF_CHECK, RF_PUSHPROPS, SET_REVNUM, and estat::st.

Referenced by ci__nondir().

Here is the caller graph for this function:

int cs__two_ch2bin ( char *  stg  )  [inline]

-.

Hex-character pair to ascii.

Faster than sscanf(). Returns -1 on error.

Definition at line 163 of file checksum.c.

References cs__hex2val().

Referenced by url___dump().

Here is the call graph for this function:

Here is the caller graph for this function:


Variable Documentation

struct t_manber_data cs___manber [static]

The Manber-structure.

Currently only a single instance of manber-hashing runs at once, so we simply use a static structure.

Definition at line 88 of file checksum.c.

Referenced by cs___update_manber(), and cs__new_manber_filter().

const char cs___mb_rd_format[] = "%*s%n %x %llu %llu\n"

The read format string for md5s.

Definition at line 94 of file checksum.c.

Referenced by cs__read_manber_hashes().

const char cs___mb_wr_format[] = "%s %08x %10llu %10llu\n"

The write format string for md5s.

Definition at line 92 of file checksum.c.

Referenced by cs___update_manber().

The precalculated CRC-table.

Definition at line 82 of file checksum.c.

Referenced by cs___end_of_block().


Generated for fsvs by  doxygen 1.5.9