FTS3TokenizerDescriptor

public struct FTS3TokenizerDescriptor

An FTS3 tokenizer, suitable for FTS3 and FTS4 table definitions:

db.create(virtualTable: "book", using: FTS4()) { t in
    t.tokenizer = .simple // FTS3TokenizerDescriptor
}

See https://www.sqlite.org/fts3.html#tokenizer

Show on GitHub

  • The “simple” tokenizer.

    db.create(virtualTable: "book", using: FTS4()) { t in
    t.tokenizer = .simple
}

    See https://www.sqlite.org/fts3.html#tokenizer

    Declaration

    Swift

    public static let simple: FTS3TokenizerDescriptor
    Show on GitHub

  • The “porter” tokenizer.

    db.create(virtualTable: "book", using: FTS4()) { t in
    t.tokenizer = .porter
}

    See https://www.sqlite.org/fts3.html#tokenizer

    Declaration

    Swift

    public static let porter: FTS3TokenizerDescriptor
    Show on GitHub

  • The “unicode61” tokenizer.

    db.create(virtualTable: "book", using: FTS4()) { t in
    t.tokenizer = .unicode61()
}

    See https://www.sqlite.org/fts3.html#tokenizer

    Declaration

    Swift

    public static func unicode61(
    diacritics: FTS3.Diacritics = .removeLegacy,
    separators: Set<Character> = [],
    tokenCharacters: Set<Character> = [])
-> FTS3TokenizerDescriptor

    Parameters

    diacritics

    By default SQLite will strip diacritics from latin characters.
    separators

    Unless empty (the default), SQLite will consider these characters as token separators.
    tokenCharacters

    Unless empty (the default), SQLite will consider these characters as token characters.
    Show on GitHub