CSCI N241 Web Design: Power Web Searching
1. Finding stuff on the web:
"The Internet is a great source of information" "You can find whatever you want on the Web" "The Internet is like the world's largest library" "All you need is to know XYZ search engine, and it's easy"

CSCI N241 Web Design: Power Web Searching
2. The previous slide was full of LIES!!!
The Internet is hard to find stuff in It is totally disorganized Finding information on the web can be extremely difficult

CSCI N241 Web Design: Power Web Searching
3. Why finding stuff is so hard
Nobody is in charge Anyone can add or delete It changes very quickly There is no index or table of contents

CSCI N241 Web Design: Power Web Searching
4. Applying STAIR to web searching
Stating the problem so it's easier to find stuff The various kinds of tools Some clever approaches Refinement strategies

CSCI N241 Web Design: Power Web Searching
5. Stating the problem
What are you REALLY trying to find? How will you know when you are close There are different strategies for different kinds of problems Is that something you are likely to get? Perhaps you can find a related site more easily

CSCI N241 Web Design: Power Web Searching
6. Finding a certain page
Sometimes you are looking for one particular page If you know its title or headlines, that might help Think of an unusual phrase within the page Think of other pages that might link to that one Try searching for the person or organization that created the page

CSCI N241 Web Design: Power Web Searching
7. Looking for a general topic
Direct search for that topic Think of key phrases used in discussion of the topic look for specialized portal sites about the topic Check newsgroups and FAQs about the topic if you can find them

CSCI N241 Web Design: Power Web Searching
8. There are several categories of tools:
Search engines Interior searches Meta-searches Portals Specialty portals Tools within the browser

CSCI N241 Web Design: Power Web Searching
9. Search Engines are databases
They contain organized information about web pages Might contain titles headlines text meta data

CSCI N241 Web Design: Power Web Searching
10. Web Bots and Spiders (spooky!!!)
Search engines often use special programs called web bots or spiders Look at a page Store it's contents in the database Collect the links Look at each of them in the same way

CSCI N241 Web Design: Power Web Searching
11. Search engines rank the results by:
exactness of the match number of times word or phrase comes up other criteria, sometimes mysterious

CSCI N241 Web Design: Power Web Searching
12. Boolean operators
And - narrows a search Or - broadens a search Not - excludes something Parentheses used for order of operation Default operation varies by browser

CSCI N241 Web Design: Power Web Searching
13. Examples of typical search engines:
Altavista Excite Lycos WebCrawler

CSCI N241 Web Design: Power Web Searching
14. Search engines are best at:
You type in a word, It looks for a page that has it You refine until you have a reasonable number of pages to search

CSCI N241 Web Design: Power Web Searching
15. Great things about search engines:
They can cover LARGE parts of the Internet Good if you know exactly what word or phrase you are looking for They might rank the results in a very useful order Some allow you to ask questions in "English" (ask Jeeves) If you are skillfull, they can be very powerful

CSCI N241 Web Design: Power Web Searching
16. Problems with search engines:
You almost always have to refine You either get 12 billion hits or zero The different engines are very inconsistent They do not all have the same syntax Boolean operators can be hard to use It takes a lot of practice and imagination to use a search engine well

CSCI N241 Web Design: Power Web Searching
17. The best strategies:
Look for unique phrases Search for combinations of words, often inside quotes Use a search engine to find a specialized portal (we'll discuss those in a bit) Learn the boolean operators for your favorite couple of engines

CSCI N241 Web Design: Power Web Searching
18. Interior searches:
The large search engines search the ENTIRE Internet (or proport to, anyway) Some large sites have their own internal search engine This is helpful when you are in the right answer but have not found the answer

CSCI N241 Web Design: Power Web Searching
19. Some interior searches:
Indiana University UITS Knowledge base Search TIME.com on Pathfinder Search my site (on Andy's home page)

CSCI N241 Web Design: Power Web Searching
20. When to use interior searches
You are at the right person or organization's site You are looking for something specific at that site

CSCI N241 Web Design: Power Web Searching
21. Meta-searches
Special search engines Accept a search request Send it to many other search engines

CSCI N241 Web Design: Power Web Searching
22. Advantages of metasearches
Give you an even larger database to work with Tell you quickly which search engine might be most promising Especially difficult for obscure searches that do not work in your 'favorite' engine

CSCI N241 Web Design: Power Web Searching
23. Disadvantages of metasearches
Use 'lowest common denominator' boolean expression syntax You don't know how your search will be determined LOTS of duplication The problem is usually you start with too many hits. Meta searches make that worse!!

CSCI N241 Web Design: Power Web Searching
24. Portals
These are the "entry" sites They usually have a categorized list of topics Each topic has a number of subtopics (hierarchy tree) Usually the topics and subtopics are chosen and organized by a human editor

CSCI N241 Web Design: Power Web Searching
25. Some examples of portal sites:
yahoo Microsoft Network Goto.com

CSCI N241 Web Design: Power Web Searching
26. What makes it a portal site?
Portals are maintained by human editors The pages on the site are (presumably) of some quality There is an attempt at organizing the pages by category

CSCI N241 Web Design: Power Web Searching
27. How to use a portal site
Go to the root Look around If you see your target, cool If not, do you see a promising sub menu? If so, go there and repeat the process If not, back up a level and repeat the process

CSCI N241 Web Design: Power Web Searching
28. Advantages of portals:
The best imitation of a table of contents on the Internet Some (notably yahoo) are de facto "who's who" sites on the Web You're less likely to run into garbage, as somebody had to put each site there Topical searches are much easier than boolean expressions

CSCI N241 Web Design: Power Web Searching
29. Disadvantages of portals:
Maybe the editor didn't like the page you want They are by necessity more limited that automated search engines They are very uneven in their coverage

CSCI N241 Web Design: Power Web Searching
30. Specialized portals
In any given topic, you will find a small group (often one) definitive expert There may be many sites, but most will link back to this one or two These sites are usually created by passionate amateurs If you can use another tool to find the right specialized portal, You will find every link you need right there

CSCI N241 Web Design: Power Web Searching
31. Advantages of specialized portals:
They are usually done by fanatics who know ALL the sites around a topic The best ones are kept up to date all the time They are often rated by content or edited to only contain the 'best' sites

CSCI N241 Web Design: Power Web Searching
32. Disadvantages of specialized portals:
They can be very difficult to find Even fanatics have jobs and spouses Sometimes they fade away without warning The page you're looking for might be 'junk' to the page editor Some topics have much better coverage than others

CSCI N241 Web Design: Power Web Searching
33. Tools within the browser
Backup button History list Search inside page

• Altavista
• Excite
• Lycos
• WebCrawler

Search engines are best at:

• You type in a word,
• It looks for a page that has it
• You refine until you have a reasonable number of pages to search

Great things about search engines:

• They can cover LARGE parts of the Internet
• Good if you know exactly what word or phrase you are looking for
• They might rank the results in a very useful order
• Some allow you to ask questions in "English" (ask Jeeves)
• If you are skillfull, they can be very powerful

Problems with search engines:

• You almost always have to refine
• You either get 12 billion hits or zero
• The different engines are very inconsistent
• They do not all have the same syntax
• Boolean operators can be hard to use
• It takes a lot of practice and imagination to use a search engine well

The best strategies:

• Look for unique phrases
• Search for combinations of words, often inside quotes
• Use a search engine to find a specialized portal (we'll discuss those in a bit)
• Learn the boolean operators for your favorite couple of engines

Interior searches:

• The large search engines search the ENTIRE Internet
• (or proport to, anyway)
• Some large sites have their own internal search engine
• This is helpful when you are in the right answer but have not found the answer

Some interior searches:

• Indiana University UITS Knowledge base
• Search TIME.com on Pathfinder
• Search my site (on Andy's home page)

When to use interior searches

• You are at the right person or organization's site
• You are looking for something specific at that site

Meta-searches

• Special search engines
• Accept a search request
• Send it to many other search engines

Advantages of metasearches

• Give you an even larger database to work with
• Tell you quickly which search engine might be most promising
• Especially difficult for obscure searches that do not work in your 'favorite' engine

Disadvantages of metasearches

• Use 'lowest common denominator' boolean expression syntax
• You don't know how your search will be determined
• LOTS of duplication
• The problem is usually you start with too many hits. Meta searches make that worse!!

Portals

• These are the "entry" sites
• They usually have a categorized list of topics
• Each topic has a number of subtopics (hierarchy tree)
• Usually the topics and subtopics are chosen and organized by a human editor

Some examples of portal sites:

• yahoo
• Microsoft Network
• Goto.com

What makes it a portal site?

• Portals are maintained by human editors
• The pages on the site are (presumably) of some quality
• There is an attempt at organizing the pages by category

How to use a portal site

• Go to the root
• Look around
• If you see your target, cool
• If not, do you see a promising sub menu?
• If so, go there and repeat the process
• If not, back up a level and repeat the process

Advantages of portals:

• The best imitation of a table of contents on the Internet
• Some (notably yahoo) are de facto "who's who" sites on the Web
• You're less likely to run into garbage, as somebody had to put each site there
• Topical searches are much easier than boolean expressions

Disadvantages of portals:

• Maybe the editor didn't like the page you want
• They are by necessity more limited that automated search engines
• They are very uneven in their coverage

Specialized portals

• In any given topic, you will find a small group (often one) definitive expert
• There may be many sites, but most will link back to this one or two
• These sites are usually created by passionate amateurs
• If you can use another tool to find the right specialized portal,
• You will find every link you need right there

Advantages of specialized portals:

• They are usually done by fanatics who know ALL the sites around a topic
• The best ones are kept up to date all the time
• They are often rated by content or edited to only contain the 'best' sites

Disadvantages of specialized portals:

• They can be very difficult to find
• Even fanatics have jobs and spouses
• Sometimes they fade away without warning
• The page you're looking for might be 'junk' to the page editor
• Some topics have much better coverage than others

Tools within the browser

• Backup button
• History list
• Search inside page