FTS3TokenizerDescriptor

public struct FTS3TokenizerDescriptor

An FTS3 tokenizer, suitable for FTS3 and FTS4 table definitions:

db.create(virtualTable: "books", using: FTS4()) { t in
    t.tokenizer = .simple // FTS3TokenizerDescriptor
}

See https://www.sqlite.org/fts3.html#tokenizer

  • The simple tokenizer.

    db.create(virtualTable: "books", using: FTS4()) { t in
    t.tokenizer = .simple
}

    See https://www.sqlite.org/fts3.html#tokenizer

    Declaration

    Swift

    public static let simple = FTS3TokenizerDescriptor("simple")
    Show on GitHub

  • The porter tokenizer.

    db.create(virtualTable: "books", using: FTS4()) { t in
    t.tokenizer = .porter
}

    See https://www.sqlite.org/fts3.html#tokenizer

    Declaration

    Swift

    public static let porter = FTS3TokenizerDescriptor("porter")
    Show on GitHub

  • The unicode61 tokenizer.

    db.create(virtualTable: "books", using: FTS4()) { t in
    t.tokenizer = .unicode61()
}

    See https://www.sqlite.org/fts3.html#tokenizer

    Declaration

    Swift

    public static func unicode61(removeDiacritics: Bool = true, separators: Set<Character> = [], tokenCharacters: Set<Character> = []) -> FTS3TokenizerDescriptor

    Parameters

    removeDiacritics

    If true (the default), then SQLite will strip diacritics from latin characters.
    separators

    Unless empty (the default), SQLite will consider these characters as token separators.
    tokenCharacters

    Unless empty (the default), SQLite will consider these characters as token characters.
    Show on GitHub