
Tech
Text Dataset Cleaner: fix messy Tamil & English datasets offline with a 1B Gemma (Hacktoberfest Week 3)
Submission for the Hacktoberfest Open-Source AI Challenge: Week 3 — Slow and Steady . Built during the challenge window, runs entirely on my laptop. Code: * github.com/vishnumur777/text-dataset-cleaner * .
The chore I kept putting off
I have a folder of Tamil and English sentences scraped from forms, chat logs and OCR dumps. They are mostly fine — the worst kind of fine:
In the data
What it should be
down loa ded the dataset
downloaded the d...
Read the full discussion on Dev.to
This article was aggregated from Dev.to. Click to join the conversation.
View on Dev.to