diff options
| author |   HsiangNianian <i@jyunko.cn> | 2025-12-30 19:14:39 +0800 |
|---|---|---|
| committer | HsiangNianian <i@jyunko.cn> | 2025-12-30 19:14:39 +0800 |
| commit | 7ac684f1f82023c6284cd7d7efde11b8dc98c149 (patch) | |
| tree | 4ac4e9fb72a4e1e2578d9fb4e9704967b052ec15 /requirements.txt | |
| parent | 12910f3a937633a25aa0de463a6edf756f2b8cdd (diff) | |
| download | base-model-7ac684f1f82023c6284cd7d7efde11b8dc98c149.tar.gz base-model-7ac684f1f82023c6284cd7d7efde11b8dc98c149.zip | |
feat: Implement TRPG NER training and inference script with robust model path detection and enhanced timestamp/speaker handling
- Added main training and inference logic in main.py, including CoNLL parsing, tokenization, and model training.
- Introduced TRPGParser class for inference with entity aggregation and special handling for timestamps and speakers.
- Developed utility functions for converting word-level CoNLL to char-level and saving datasets in various formats.
- Added ONNX export functionality for the trained model.
- Created a comprehensive requirements.txt and updated pyproject.toml with necessary dependencies.
- Implemented tests for ONNX inference to validate model outputs.
Diffstat (limited to 'requirements.txt')
| -rw-r--r-- | requirements.txt | 136 |
1 files changed, 136 insertions, 0 deletions
diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..bd6d100 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,136 @@ +accelerate==1.12.0 +aiofiles==24.1.0 +aiohappyeyeballs==2.6.1 +aiohttp==3.13.2 +aiosignal==1.4.0 +annotated-doc==0.0.4 +annotated-types==0.7.0 +anyio==4.12.0 +asttokens==3.0.1 +attrs==25.4.0 +brotli==1.2.0 +certifi==2025.11.12 +charset-normalizer==3.4.4 +click==8.3.1 +coloredlogs==15.0.1 +comm==0.2.3 +datasets==4.4.2 +debugpy==1.8.19 +decorator==5.2.1 +dill==0.4.0 +executing==2.2.1 +fastapi==0.128.0 +ffmpy==1.0.0 +filelock==3.20.1 +flatbuffers==25.12.19 +frozenlist==1.8.0 +fsspec==2025.12.0 +gradio==6.2.0 +gradio-client==2.0.2 +groovy==0.1.2 +h11==0.16.0 +hf-xet==1.2.0 +httpcore==1.0.9 +httpx==0.28.1 +huggingface-hub==0.36.0 +humanfriendly==10.0 +idna==3.11 +ipykernel==7.1.0 +ipython==9.8.0 +ipython-pygments-lexers==1.1.1 +jedi==0.19.2 +jinja2==3.1.6 +joblib==1.5.3 +jupyter-client==8.7.0 +jupyter-core==5.9.1 +markdown-it-py==4.0.0 +markupsafe==3.0.3 +matplotlib-inline==0.2.1 +maturin==1.10.2 +mdurl==0.1.2 +ml-dtypes==0.5.4 +mpmath==1.3.0 +multidict==6.7.0 +multiprocess==0.70.18 +nest-asyncio==1.6.0 +networkx==3.6.1 +numpy==2.4.0 +nvidia-cublas-cu12==12.8.4.1 +nvidia-cuda-cupti-cu12==12.8.90 +nvidia-cuda-nvrtc-cu12==12.8.93 +nvidia-cuda-runtime-cu12==12.8.90 +nvidia-cudnn-cu12==9.10.2.21 +nvidia-cufft-cu12==11.3.3.83 +nvidia-cufile-cu12==1.13.1.3 +nvidia-curand-cu12==10.3.9.90 +nvidia-cusolver-cu12==11.7.3.90 +nvidia-cusparse-cu12==12.5.8.93 +nvidia-cusparselt-cu12==0.7.1 +nvidia-ml-py==13.590.44 +nvidia-nccl-cu12==2.27.5 +nvidia-nvjitlink-cu12==12.8.93 +nvidia-nvshmem-cu12==3.3.20 +nvidia-nvtx-cu12==12.8.90 +onnx==1.20.0 +onnx-ir==0.1.13 +onnxruntime==1.23.2 +onnxscript==0.5.7 +orjson==3.11.5 +packaging==25.0 +pandas==2.3.3 +parso==0.8.5 +pexpect==4.9.0 +pillow==12.0.0 +platformdirs==4.5.1 +prompt-toolkit==3.0.52 +propcache==0.4.1 +protobuf==6.33.2 +psutil==7.2.1 +ptyprocess==0.7.0 +pure-eval==0.2.3 +pyarrow==22.0.0 +pydantic==2.12.5 +pydantic-core==2.41.5 +pydub==0.25.1 +pygments==2.19.2 +pynvml==13.0.1 +python-dateutil==2.9.0.post0 +python-multipart==0.0.21 +pytz==2025.2 +pyyaml==6.0.3 +pyzmq==27.1.0 +regex==2025.11.3 +requests==2.32.5 +rich==14.2.0 +safehttpx==0.1.7 +safetensors==0.7.0 +scikit-learn==1.8.0 +scipy==1.16.3 +semantic-version==2.10.0 +seqeval==1.2.2 +setuptools==80.9.0 +shellingham==1.5.4 +six==1.17.0 +stack-data==0.6.3 +starlette==0.50.0 +sympy==1.14.0 +threadpoolctl==3.6.0 +tokenizers==0.22.1 +tomlkit==0.13.3 +torch==2.9.1 +torchaudio==2.9.1+cpu +torchvision==0.24.1+cpu +tornado==6.5.4 +tqdm==4.67.1 +traitlets==5.14.3 +transformers==4.57.3 +triton==3.5.1 +typer==0.21.0 +typing-extensions==4.15.0 +typing-inspection==0.4.2 +tzdata==2025.3 +urllib3==2.6.2 +uvicorn==0.40.0 +wcwidth==0.2.14 +xxhash==3.6.0 +yarl==1.22.0 |