https://raychi.site/tags/huggingface/Recent content in Huggingface on Ray's BlogHugozh-CNCopyright © 2025, RayThu, 28 Nov 2024 00:00:00 +0800https://raychi.site/huggingface/Thu, 28 Nov 2024 00:00:00 +0800https://raychi.site/huggingface/<h2 id="什么是hugging-face">什么是hugging face？🤔</h2> <h2 id="如何使用">如何使用？🔧</h2> <h3 id="hugging-face模型下载">hugging face模型下载</h3> <blockquote> <p>首先安装相关库</p> <div class="highlight"><pre tabindex="0" style="color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-bash" data-lang="bash"><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">1</span><span>pip install -U huggingface_hub </span></span></code></pre></div></blockquote> <p>然后编写一个python文件，如下</p> <div class="highlight"><pre tabindex="0" style="color:#e6edf3;background-color:#0d1117;-moz-tab-size:4;-o-tab-size:4;tab-size:4;"><code class="language-python" data-lang="python"><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 1</span><span><span style="color:#8b949e;font-style:italic"># 设置环境变量</span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 2</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 3</span><span><span style="color:#ff7b72">import</span> <span style="color:#ff7b72">os</span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 4</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 5</span><span>os<span style="color:#ff7b72;font-weight:bold">.</span>environ[<span style="color:#a5d6ff">&#34;HF_ENDPOINT&#34;</span>] <span style="color:#ff7b72;font-weight:bold">=</span> <span style="color:#a5d6ff">&#34;https://hf-mirror.com&#34;</span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 6</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 7</span><span><span style="color:#ff7b72">from</span> <span style="color:#ff7b72">huggingface_hub</span> <span style="color:#ff7b72">import</span> hf_hub_download </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 8</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681"> 9</span><span><span style="color:#8b949e;font-style:italic"># 举例</span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">10</span><span>repo_id <span style="color:#ff7b72;font-weight:bold">=</span> <span style="color:#a5d6ff">&#34;Duffy/Llama-820m&#34;</span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">11</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">12</span><span>file_list <span style="color:#ff7b72;font-weight:bold">=</span> [ </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">13</span><span> <span style="color:#a5d6ff">&#34;.gitattributes&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">14</span><span> <span style="color:#a5d6ff">&#34;README.md&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">15</span><span> <span style="color:#a5d6ff">&#34;config.json&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">16</span><span> <span style="color:#a5d6ff">&#34;generation_config.json&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">17</span><span> <span style="color:#a5d6ff">&#34;xxx.py&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">18</span><span> <span style="color:#a5d6ff">&#34;pytorch_model.bin&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">19</span><span> <span style="color:#a5d6ff">&#34;special_tokens_map.json&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">20</span><span> <span style="color:#a5d6ff">&#34;tokenizer.json&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">21</span><span> <span style="color:#a5d6ff">&#34;tokenizer_config.json&#34;</span>, </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">22</span><span>] </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">23</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">24</span><span>local_dir <span style="color:#ff7b72;font-weight:bold">=</span> <span style="color:#a5d6ff">&#34;./your_model_file_dir&#34;</span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">25</span><span> </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">26</span><span><span style="color:#ff7b72">for</span> file_name <span style="color:#ff7b72;font-weight:bold">in</span> file_list: </span></span><span style="display:flex;"><span style="white-space:pre;-webkit-user-select:none;user-select:none;margin-right:0.4em;padding:0 0.4em 0 0.4em;color:#6e7681">27</span><span> hf_hub_download(repo_id<span style="color:#ff7b72;font-weight:bold">=</span>repo_id, filename<span style="color:#ff7b72;font-weight:bold">=</span>file_name, local_dir<span style="color:#ff7b72;font-weight:bold">=</span>local_dir) </span></span></code></pre></div><h2 id="hugging-face-hub">Hugging Face Hub</h2> <p>Hugging Face Hub 是一个集成模型、数据集、和其他机器学习资源的平台，它使得分享和使用模型变得非常方便。Hub 提供了易于使用的 API、GitHub 风格的模型管理，并且是目前许多 NLP 和计算机视觉任务中最流行的模型资源。</p>