Token Splitter 节点#
¥Token Splitter node
令牌分割器节点首先将原始文本字符串转换为 BPE 令牌，然后将这些令牌分割成块，最后将单个块内的令牌转换回文本。
¥The Token Splitter node splits a raw text string by first converting the text into BPE tokens, then splits these tokens into chunks and converts the tokens within a single chunk back into text.
本页包含词元分割器节点的节点参数以及更多资源的链接。
¥On this page, you'll find the node parameters for the Token Splitter node, and links to more resources.
Parameter resolution in sub-nodes
Sub-nodes behave differently to other nodes when processing multiple items using an expression.
Most nodes, including root nodes, take any number of items as input, process these items, and output the results. You can use expressions to refer to input items, and the node resolves the expression for each item in turn. For example, given an input of five name values, the expression {{ $json.name }} resolves to each name in turn.
In sub-nodes, the expression always resolves to the first item. For example, given an input of five name values, the expression {{ $json.name }} always resolves to the first name.
节点参数#
¥Node parameters
- 块大小：输入每个数据块的字符数。
¥Chunk Size: Enter the number of characters in each chunk.
- 块重叠：输入数据块之间的重叠比例。
¥Chunk Overlap: Enter how much overlap to have between chunks.
模板和示例#
¥Templates and examples
相关资源#
¥Related resources
请参阅 LangChain 的词元文档 和 LangChain 的文本分割器文档 获取有关服务的更多信息。
¥Refer to LangChain's token documentation and LangChain's text splitter documentation for more information about the service.
View n8n's Advanced AI documentation.