FTS5CustomTokenizer

public protocol FTS5CustomTokenizer : FTS5Tokenizer

The protocol for custom FTS5 tokenizers.

  • The name of the tokenizer; should uniquely identify your custom tokenizer.

    Declaration

    Swift

    static var name: String { get }
  • Creates a custom tokenizer.

    The arguments parameter is an array of String built from the CREATE VIRTUAL TABLE statement. In the example below, the arguments will be ["arg1", "arg2"].

    CREATE VIRTUAL TABLE document USING fts5(
        tokenize='custom arg1 arg2'
    )
    

    Declaration

    Swift

    init(db: Database, arguments: [String]) throws

    Parameters

    db

    A Database connection

    arguments

    An array of string arguments

  • Creates an FTS5 tokenizer descriptor.

    class MyTokenizer : FTS5CustomTokenizer { ... }
    
    db.create(virtualTable: "book", using: FTS5()) { t in
        let tokenizer = MyTokenizer.tokenizerDescriptor(arguments: ["unicode61", "remove_diacritics", "0"])
        t.tokenizer = tokenizer
    }
    

    Declaration

    Swift

    public static func tokenizerDescriptor(arguments: [String] = []) -> FTS5TokenizerDescriptor