• subscribe
June 13, 2000 10:39 AM

Sample Full-Text Search Engine

SQL Server Pro
InstantDoc ID #8967

Suppose you want a table to hold all content for your dynamically populated Web site. In the following example, I'll create a simple SQL Server table and a full-text search engine to run against it. Following is the content table:

Create table content (
Pagename		varchar(20) not null primary key,
URL			archar(30) not null,
Description		text null,
Keywords		varchar(4000) null)

Insert the following values.

INSERT content values ("home.asp","home.asp","This is the home page","home,SQL")
GO
INSERT content values ("pagetwo.asp","/page2/pagetwo.asp","NT Magazine is great","second")
GO
INSERT content values ("pagethree.asp","/page3/pagethree.asp","SQL Magazine is the greatest","third")
GO 

The table will look like this:

Pagename URL Description Keywords
-------- --------- --------------------- ---------
home.asp /home.asp This is the home page home, SQL
Pagetwo.asp /page2/page2.asp NT Magazine is great second
Pagethree.asp /page3/page3.asp SQL Magazine is the greatest third

Next, review the full-text enabling procedures, enable all columns for full-text searching, and populate the index.

Let's practice querying this table using some keyword searches. Open SQL Server 7.0's Query Analyzer, and try some of the following queries.

Query 1 (FREETEXT)

SELECT * FROM content WHERE freetext(*,"home") 
Result
Pagename URL Description Keywords
-------- --------- --------------------- ---------
home.asp /home.asp This is the home page home, SQL

This queries all full-text-enabled columns in the content table for the string "home."

Query 2 (FREETEXT)

SELECT * FROM content WHERE freetext(description,"Magazine")
Result
Pagename URL Description Keywords
-------- --------- --------------------- ---------
Pagetwo.asp /page2/page2.asp NT Magazine is great second
Pagethree.asp /page3/page3.asp SQL Magazine is the greatest third

This only searches the Description column and returns all matches for the string "Magazine."

Query 3 (FREETEXT)

SELECT * FROM content WHERE freetext(description,"SQL Mag")
Result
Pagename URL Description Keywords
-------- --------- --------------------- ---------
Pagethree.asp /page3/page3.asp SQL Magazine is the greatest third

Although this appears to search on the string "SQL Mag," it actually searches on "SQL" or "Mag."

Query 4 (FREETEXT)

SELECT * FROM content WHERE freetext(description,"the")

Result

Server: Msg 7619, Level 16, State 1, Line 1

The query contains only ignored words; we've queried a noise word here. You'll find "the" in the noise words file at \MSSQL7\FTDATA\SQLSERVER\CONFIG.

Query 5 (CONTAINS)

SELECT * FROM content WHERE contains(*,"home")
Pagename URL Description Keywords
-------- --------- --------------------- ---------
home.asp /home.asp This is the home page home, SQL

Like the Freetext query, this searches all full-text-enabled columns for the keyword "home."

Query 6 (CONTAINS)

SELECT * FROM content WHERE contains(Description,'  "Magaz*"  ')
Pagename URL Description Keywords
-------- --------- --------------------- ---------
Pagetwo.asp /page2/page2.asp NT Magazine is great second
Pagethree.asp /page3/page3.asp SQL Magazine is the greatest third

This statement queries the Description column for a word beginning with "Magaz." Note that the asterisk acts as a wildcard or placeholder, just as the percent sign (%) does with the LIKE keyword. (To make this work, you need to use single quotes on either side of the double quotes.)

Query 7 (CONTAINS)

SELECT * FROM content WHERE contains(Description,'  "*azine"  ')
Pagename URL Description Keywords
-------- --------- --------------------- ---------

This search yields no results. You can't use an asterisk as a placeholder for a prefix.

Query 8 (CONTAINS)

SELECT * FROM content WHERE contains(Description,'  "Magazine" Or "Great"  ')
Pagename URL Description Keywords
-------- --------- --------------------- ---------
Pagetwo.asp /page2/page2.asp NT Magazine is great second
Pagethree.asp /page3/page3.asp SQL Magazine is the greatest third

This full-text scan uses OR so that you can search for "Magazine" or "Great"; it also works with AND and AND NOT. (Again, note the single quotes around the search criteria.)

Query 9 (CONTAINS)

SELECT * FROM content WHERE CONTAINS(description, 'NT NEAR great')
Pagename URL Description Keywords
-------- --------- --------------------- ---------
Pagetwo.asp /page2/page2.asp NT Magazine is great second

This search on the Description column finds all rows where "NT" is near "great".

Query 10 (CONTAINS)

SELECT * FROM content WHERE contains(description, ' formsof (inflectional, great) ')
Pagename URL Description Keywords
-------- --------- --------------------- ---------
Pagetwo.asp /page2/page2.asp NT Magazine is great second
Pagethree.asp /page3/page3.asp SQL Magazine is the greatest third

This statement returns all results for "great," "greatest," "greater," and so on.



ARTICLE TOOLS

Comments
  • Matthew
    5 years ago
    Mar 17, 2007

    And I subscribed just to read the rest of this????

    My mistake!

    I read the first couple of paragraphs as a visitor and thought it might actually go somewhere so I subscribed ... turns out it's a very basic rehash of info that's readily available in a dozen other places.

    If this is what's on offer ... UNSUBSCRIBE ME

  • Anonymous User
    7 years ago
    Jun 23, 2005

    In order to search and highlight you'll need to do what I did, and build your own routine to generate inflectional forms and word variants to pass to Microsoft Search - you'd be surprised how easy it is to produce better results than the native search engine using 3 relatively simple algorithms including Soundex, Levenstine and Porter's Stemmer.

  • Anonymous User
    7 years ago
    Feb 28, 2005

    This is one of the greatest articles for learning about something, that I have read. Its clear and easy to understand, wish more tutorials were like this one.
    Keep it up!

  • Anonymous User
    7 years ago
    Jan 11, 2005

    I want to highlight all occurrences of the search within a document - is there a way to read through the text and highlight each occurrence of the search string?

  • Victor Unda
    8 years ago
    Mar 05, 2004

    In the article does not mention that SQL Server Fulltext could be not "great, "greatest" and greater".
    The article need to considerate there are another implication before to create a fulltext index search as well as the process to update, delete, insert and make changes to the database “table or tables”.
    The use of FREETEXT, my personal experiences such as "search engine" are difficult. SQL Server take to long to update the data such as added, deleted or updating. In addition, if you have clients that they are depending to update their own data every day, you may have to considerer to find another options.
    By Victor Unda
    Undaconsulting.com

You must log on before posting a comment.

Are you a new visitor? Register Here
  • SP1?
    I know there is a SP1 for SQL 2008 R2 available....and there is a "feature pack" as well... ...
  • SQL database mirroring
    I have SQL Server 2008 R2 Enterprise 64bit on Windows 2008 R2 Enterprise 64bit.  Each SQL Server has...
  • Dell Compellent Disk Drive
    Does anybody has experience with Dell Compellent Disk Drive? Basically, this system manages all disk...
  • Sql server performance tuning
    I need to find a tool that help me to optimize sql server,queries,improve the performance and solve ...