Remove Duplicate Lines is a browser-based text tool that keeps the first occurrence of every line in your paste and removes every later repeat, with a 1,000,000-UTF-16-code-unit local limit and optional case and edge-whitespace toggles. The algorithm walks the lines in source order, derives a comparison key for each, checks that key against a Set of already-seen keys, and appends only the first unseen line, so the operation is deterministic for the input and the chosen options. Rhino 3D ships with its own SelDup command for catching coincident geometry, but the text side of a Rhino workflow produces repeated lines too — layer name dumps, log output from scripts, exported identifier columns, and tag lists copied from a Grasshopper panel. Paste the list into the input area, pick strict or normalized comparison, run the tool, read the input, retained, and removed counts to verify the result, then paste the deduped output back into your Rhino file or downstream document.
When You Need to Deduplicate Lines from Rhino Text
Rhino 3D is primarily a geometry engine, but most real projects move a lot of plain text alongside the curves and surfaces. Common cases that produce repeating text lines include:
- Layer name exports from a layer table dump or a script like GetLayerNames
- Tag and block-name lists copied from Grasshopper panels
- Filename lists generated by batch export to DWG, PDF, or STL
- Material, hatch pattern, or dimension style names that a template file has duplicated
- Identifier columns copied from a CSV of object IDs, coordinates, or notes
- Script log output where a Grasshopper definition prints the same value to the panel on every iteration
For each of these, the practical problem is identical: a long list in a text buffer with no easy way to collapse the repeats. Pasting into a spreadsheet and wrestling with formulas is slow. Sorting then deleting visually is error-prone. Running the lines through a local line-by-line deduplicator is the cleanest fix, especially when the file lives on a locked-down workstation or contains identifiers you do not want to upload to a network service.
How to Remove Duplicate Lines from Rhino in the Browser
The workflow is short and stays entirely on your local machine. Follow these steps when you have a Rhino text list to clean up.
- In Rhino, run the command that produces your text list. Layer-table dumps, List output, Grasshopper panel output, and script console logs all copy as plain text.
- Paste the list into the input area of the Remove Duplicate Lines tool, one value per line.
- Choose how comparison should work: leave both options off for strict matching, or turn on ignore case, ignore edge whitespace, or both.
- Run the tool. It walks the lines in source order, computes a comparison key for each line, retains the first line whose key has not appeared, and joins the survivors with LF.
- Read the input, retained, and removed counts in the result panel and confirm the numbers match what you expected.
- Copy the deduplicated output and paste it back into Rhino, your script, your CSV, or your downstream file.
Choosing Between Strict and Normalized Comparison
Strict mode compares each complete line exactly. Capitalization, leading and trailing spaces, tabs, punctuation, Unicode characters, and internal spacing all participate. Under strict comparison, Apple and apple are different, and a line that ends with one trailing space differs from the same visible letters without that space. This mode is the safest choice when whitespace or capitalization carries meaning, for instance when layer names in Rhino embed material codes, view names, or Grasshopper paths whose case is load-bearing.
Ignore English letter case builds the comparison key with the browser's English-locale lowercase operation. It keeps the spelling and capitalization of the first occurrence rather than rewriting the output. That makes it a practical match for ordinary English lists, but it is not accent folding, transliteration, Unicode normalization, fuzzy matching, stemming, or language-aware collation. German sharp s is not automatically treated as ss, and accented letters stay distinct from their unaccented cousins.
Ignore edge whitespace trims the comparison key at both ends. It does not trim the retained output. If the first occurrence is surrounded by spaces and a later line contains the same visible value without them, the spaced first occurrence is preserved exactly and the later line is removed. Internal spaces and tabs still matter. Combining case and trim options applies trimming first and the English-locale lowercase operation second.
| Mode | What counts as a duplicate | Best for Rhino text |
|---|---|---|
| Strict (default) | Lines match exactly including case and whitespace | Layer or material codes where case matters |
| Ignore English letter case | Apple and apple match | Human-readable layer or tag names |
| Ignore edge whitespace | Leading and trailing spaces are stripped before comparison | Lists pulled from a copy-paste with stray padding |
| Both options combined | Trim first, then lowercase | Messy exports where both vary |
What the Tool Will and Will Not Do for Rhino Data
Understanding the boundary keeps the work honest. The tool will, on Rhino text exports:
- Keep the first occurrence of each comparison key exactly as written
- Recognize Windows CRLF, classic carriage return, and line feed as line separators, with a CRLF pair counted as one boundary, not two
- Report input, retained, and removed line counts
- Process everything locally in the browser, so no list, log, or layer export leaves your machine
- Treat blank lines as valid values; under strict comparison the first empty line is kept and later empty lines are removed as duplicates, while with ignore edge whitespace a whitespace-only line and an empty line share the same key so only the first of either remains
It will not:
- Parse CSV quoting, JSON arrays, code syntax, or Rhino's own locale-specific record format
- Search the output again, merge near-duplicates, count word frequency, sort the list, or infer headers
- Pick the newest, longest, most frequent, or otherwise preferred duplicate when more than one occurrence could be retained
- Mutate the original Rhino file or database, so always keep the source layer table or log available until the result is checked
Input is bounded at 1,000,000 UTF-16 code units. The exact boundary is accepted and one code unit beyond it is rejected before the tool splits lines or allocates comparison keys, so very large layer dumps need to be chunked. Empty input is rejected rather than reporting a misleading one-line result. Editing the text or either option clears the previous result, so stale output cannot be confused with current settings.
After the Dedup: Counting and Cleanup Steps
After the run, a few quick checks tighten the result before you paste it back into Rhino. First, review whether the first occurrence is the one you want to keep. If a later entry had a fix applied, sort your input first or hand-edit before pasting; the tool always trusts the first line for each comparison key. Second, decide what to do with blank lines. If your Grasshopper log or layer dump contains empty entries from skipped iterations, those are deduplicated by their empty key under strict mode but still appear in the output. Use the Remove Empty Lines tool afterward if the output should contain no blank values.
Third, if the export mixed two identifier columns into one paste, separate them first with the Column to Comma Separated List tool or by a regex, then dedupe, then rejoin. The tool does not understand quoted CSV, so a value containing a comma is treated as one line of plain text including the comma. For a quick sanity check on the cleaned list, run the Line Counter so you can see total, non-blank, and blank line counts side by side and confirm the deduped numbers match the input minus removed. If you need to confirm a specific identifier did not disappear, paste the original and the deduped output into the Compare Two Lists tool to see exactly which entries were removed.
For multi-column data, ask whether deduplicating by a visible line is the right rule at all. For business records imported into Rhino as text lists, deduplication by a stable database identifier is usually stronger than deduplication by a human-readable label. Confirm the comparison rule before deleting source data, and keep the original Rhino file available until the cleaned list is verified.
For a deeper look, see Remove Duplicate Words in Excel Without Formulas.
For a deeper look, see Remove Empty Lines in Visual Studio Code Safely.