23 lines
8.4 KiB
HTML
23 lines
8.4 KiB
HTML
<!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><meta name="generator" content="rustdoc"><meta name="description" content="ZDICT_trainFromBuffer(): Train a dictionary from an array of samples. Redirect towards ZDICT_optimizeTrainFromBuffer_fastCover() single-threaded, with d=8, steps=4, f=20, and accel=1. Samples must be stored concatenated in a single flat buffer `samplesBuffer`, supplied with an array of sizes `samplesSizes`, providing the size of each sample, in order. The resulting dictionary will be saved into `dictBuffer`. @return: size of dictionary stored into `dictBuffer` (<= `dictBufferCapacity`) or an error code, which can be tested with ZDICT_isError(). Note: Dictionary training will fail if there are not enough samples to construct a dictionary, or if most of the samples are too small (< 8 bytes being the lower limit). If dictionary training fails, you should use zstd without a dictionary, as the dictionary would’ve been ineffective anyways. If you believe your samples would benefit from a dictionary please open an issue with details, and we can look into it. Note: ZDICT_trainFromBuffer()’s memory usage is about 6 MB. Tips: In general, a reasonable dictionary has a size of ~ 100 KB. It’s possible to select smaller or larger size, just by specifying `dictBufferCapacity`. In general, it’s recommended to provide a few thousands samples, though this can vary a lot. It’s recommended that total size of all samples be about ~x100 times the target size of dictionary."><meta name="keywords" content="rust, rustlang, rust-lang, ZDICT_trainFromBuffer"><title>ZDICT_trainFromBuffer in zstd_sys - Rust</title><link rel="preload" as="font" type="font/woff2" crossorigin href="../SourceSerif4-Regular.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../FiraSans-Regular.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../FiraSans-Medium.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../SourceCodePro-Regular.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../SourceSerif4-Bold.ttf.woff2"><link rel="preload" as="font" type="font/woff2" crossorigin href="../SourceCodePro-Semibold.ttf.woff2"><link rel="stylesheet" type="text/css" href="../normalize.css"><link rel="stylesheet" type="text/css" href="../rustdoc.css" id="mainThemeStyle"><link rel="stylesheet" type="text/css" href="../ayu.css" disabled><link rel="stylesheet" type="text/css" href="../dark.css" disabled><link rel="stylesheet" type="text/css" href="../light.css" id="themeStyle"><script id="default-settings" ></script><script src="../storage.js"></script><script src="../crates.js"></script><script defer src="../main.js"></script>
|
||
<noscript><link rel="stylesheet" href="../noscript.css"></noscript><link rel="alternate icon" type="image/png" href="../favicon-16x16.png"><link rel="alternate icon" type="image/png" href="../favicon-32x32.png"><link rel="icon" type="image/svg+xml" href="../favicon.svg"></head><body class="rustdoc fn"><!--[if lte IE 11]><div class="warning">This old browser is unsupported and will most likely display funky things.</div><![endif]--><nav class="sidebar"><div class="sidebar-menu" role="button">☰</div><a class="sidebar-logo" href="../zstd_sys/index.html"><div class="logo-container"><img class="rust-logo" src="../rust-logo.png" alt="logo"></div>
|
||
</a><div class="sidebar-elems"><h2 class="location">Other items in<br><a href="index.html">zstd_sys</a></h2><div id="sidebar-vars" data-name="ZDICT_trainFromBuffer" data-ty="fn" data-relpath=""></div><script defer src="sidebar-items.js"></script></div></nav><main><div class="width-limiter"><div class="sub-container"><a class="sub-logo-container" href="../zstd_sys/index.html"><img class="rust-logo" src="../rust-logo.png" alt="logo"></a><nav class="sub"><div class="theme-picker"><button id="theme-picker" aria-label="Pick another theme!" aria-haspopup="menu" title="themes"><img width="18" height="18" alt="Pick another theme!" src="../brush.svg"></button><div id="theme-choices" role="menu"></div></div><form class="search-form"><div class="search-container"><div><select id="crate-search"><option value="All crates">All crates</option></select><input class="search-input" name="search" autocomplete="off" spellcheck="false" placeholder="Click or press ‘S’ to search, ‘?’ for more options…" type="search"></div><button type="button" id="help-button" title="help">?</button><a id="settings-menu" href="../settings.html" title="settings"><img width="18" height="18" alt="Change settings" src="../wheel.svg"></a></div></form></nav></div><section id="main-content" class="content"><h1 class="fqn"><span class="in-band">Function <a href="index.html">zstd_sys</a>::<wbr><a class="fn" href="#">ZDICT_trainFromBuffer</a><button id="copy-path" onclick="copy_path(this)" title="Copy item path to clipboard"><img src="../clipboard.svg" width="19" height="18" alt="Copy item path"></button></span><span class="out-of-band"><span id="render-detail"><a id="toggle-all-docs" href="javascript:void(0)" title="collapse all docs">[<span class="inner">−</span>]</a></span><a class="srclink" href="../src/zstd_sys/bindings_std.rs.html#969-975" title="goto source code">[src]</a></span></h1><div class="docblock item-decl"><pre class="rust fn"><code>pub unsafe extern "C" fn ZDICT_trainFromBuffer(<br> dictBuffer: <a class="primitive" href="https://doc.rust-lang.org/1.59.0/std/primitive.pointer.html">*mut </a><a class="enum" href="https://doc.rust-lang.org/1.59.0/core/ffi/enum.c_void.html" title="enum core::ffi::c_void">c_void</a>, <br> dictBufferCapacity: <a class="primitive" href="https://doc.rust-lang.org/1.59.0/std/primitive.usize.html">usize</a>, <br> samplesBuffer: <a class="primitive" href="https://doc.rust-lang.org/1.59.0/std/primitive.pointer.html">*const </a><a class="enum" href="https://doc.rust-lang.org/1.59.0/core/ffi/enum.c_void.html" title="enum core::ffi::c_void">c_void</a>, <br> samplesSizes: <a class="primitive" href="https://doc.rust-lang.org/1.59.0/std/primitive.pointer.html">*const </a><a class="primitive" href="https://doc.rust-lang.org/1.59.0/std/primitive.usize.html">usize</a>, <br> nbSamples: <a class="type" href="https://doc.rust-lang.org/1.59.0/std/os/raw/type.c_uint.html" title="type std::os::raw::c_uint">c_uint</a><br>) -> <a class="primitive" href="https://doc.rust-lang.org/1.59.0/std/primitive.usize.html">usize</a></code></pre></div><details class="rustdoc-toggle top-doc" open><summary class="hideme"><span>Expand description</span></summary><div class="docblock"><p>ZDICT_trainFromBuffer():
|
||
Train a dictionary from an array of samples.
|
||
Redirect towards ZDICT_optimizeTrainFromBuffer_fastCover() single-threaded, with d=8, steps=4,
|
||
f=20, and accel=1.
|
||
Samples must be stored concatenated in a single flat buffer <code>samplesBuffer</code>,
|
||
supplied with an array of sizes <code>samplesSizes</code>, providing the size of each sample, in order.
|
||
The resulting dictionary will be saved into <code>dictBuffer</code>.
|
||
@return: size of dictionary stored into <code>dictBuffer</code> (<= <code>dictBufferCapacity</code>)
|
||
or an error code, which can be tested with ZDICT_isError().
|
||
Note: Dictionary training will fail if there are not enough samples to construct a
|
||
dictionary, or if most of the samples are too small (< 8 bytes being the lower limit).
|
||
If dictionary training fails, you should use zstd without a dictionary, as the dictionary
|
||
would’ve been ineffective anyways. If you believe your samples would benefit from a dictionary
|
||
please open an issue with details, and we can look into it.
|
||
Note: ZDICT_trainFromBuffer()’s memory usage is about 6 MB.
|
||
Tips: In general, a reasonable dictionary has a size of ~ 100 KB.
|
||
It’s possible to select smaller or larger size, just by specifying <code>dictBufferCapacity</code>.
|
||
In general, it’s recommended to provide a few thousands samples, though this can vary a lot.
|
||
It’s recommended that total size of all samples be about ~x100 times the target size of dictionary.</p>
|
||
</div></details></section><section id="search" class="content hidden"></section></div></main><div id="rustdoc-vars" data-root-path="../" data-current-crate="zstd_sys" data-themes="ayu,dark,light" data-resource-suffix="" data-rustdoc-version="1.59.0 (9d1b2106e 2022-02-23)" ></div>
|
||
</body></html> |