A Post-Gutenberg Student Research Project
Suzanne J. Smith
Abstract
Students researched topics in Management and Organizational Behavior using the World Wide Web. They identified five sites representing their topic and categorized them as being either an example of a management theory/research or as being an application of management practice. Each topic included sites of both types. The students then summarized and evaluated the content of each site and wrote up an analysis of the web as a research tool. The students and the instructor found that while the web has great power as a research tool, harnessing that power requires a strategic understanding of search engine services and availability of site maps within each discipline.
How do undergraduate students learn how to do research on the Internet? In Management and Organizational Behavior class, an assignment given during the fall semester of 1996 attempted to break the paradigm of traditional research papers by requiring students to analyze web sites relevant to a management topic.
As the World Wide Web continues to add journals, company background information, and sites dedicated to contributors to the management field, the project's goal was to require students to become familiar with doing research on the web. Students found five different web sites . . representing a topic in one of two ways:
1. A principle or fact-based site that defined terms or gave background information about a management topic
2. A practical application of the principle in an organization's site or journal article about organizational practice.
After surfing for topics, students narrowed the search down to one topic. They then proceeded to use search engines, business directories, and other links to identify five sites representing their topic. While this sounds like barely an afternoon's work, several students spent as much as 20 -30 hours just surfing. And, as anyone who surfs knows well, time warps on the web.
Search Engine Failure: When Less Is More
Perhaps the most frustrating element of the exercise was narrowing down the topic using search engines. Although the instructor and the recommended text illustrated several ways of specifying and narrowing searches, there were many cases where search engines just couldn't filter out worthless sites. Students found hits numbering in the tens or hundreds of thousands. Ranking by relevance helped in some cases, but in others, the search service just couldn't understand what students meant by relevant.
Search services using literal string quotation marks such as "delight the customer" (Alta Vista, Infoseek) usually narrowed searches better than those that searched for every word in a string (Lycos, Magellan, Excite). A one-word topic such as "ethics" needed more specification. Table 1 reflects the wide range in number of hits based on the key word search system. Each search engine or directory has its own system of searching. Learning the system that works best for each study can be valuable. Each search system also has its own way of formatting its hits, and some provide valuable information such as the date the page was last updated.
Just because the search found a large number of hits didn't mean the hits were at all relevant to the topic at hand or even ranked according to the relevancy of the searcher. For example, the first hit under "delight the customer" in Magellan is in German. Another is for a painting in a museum with the word "delight" in it. Very few hits seemed relevant to the concept of delighting the customer in Total Quality Management.
Table 1. A comparison of hits using six search services.
-
| Search Engine | "delight the customer" | ethics |
| Yahoo | 63 | 245 |
| Magellan | 5178 | 13,554 |
| Infoseek | 111 | 115,623 |
| Alta Vista | 63 | 200,000 |
| Lycos | 32,579 | 20,027 |
| Excite | 4,314 | 166,033 |
Link Rot
A second disturbing finding was that terrific links available today might be gone tomorrow. Some of the more responsible students, unfortunately, were not rewarded for starting their projects early. After they had identified several good sites, weeks elapsed before they actually evaluated and wrote up the site. By then, one discouraged student reported that three of her best sites became unavailable. Back to the drawing board.
Analysis and Write-Up
Each student chose five sites representing management principles and practice, and the one to two page report of each site included
The final element of the project included a write-up of the entire procedure. Students addressed the questions
For most students, this project was their first assignment done totally on the web. For some, it was their first experience on the Internet. Some suggested that search engines could be improved or refined to exclude sites such as another class's syllabus or lecture notes. Many had trouble finding examples of management practices on the web. While most organizations might want their potential customers to know all about their contributions to ethics and social responsibility, many don't post human resources management policies on the Internet. Internal policies are normally posted on an intranet accessible only from within the organization's firewall.
Lessons Learned
Alrhough nearly all underestimated the amount of time the project would take, students appreciated the opportunity to learn more about searching on the web. In the process of searching for management topics, they learned that organizations really do use the concepts presented in class.
The project held lessons for the instructor as well. In the future, such projects should require students to spend less time searching and more time learning about and linking to web resources that can lead more quickly to the right places. Students and faculty must understand the capabilities and limitations of searching via search engines. Identifying pages with links to relevant sources in an academic field is a good alternative. Business Researcher's Interest (www.brint.com) is an example of a searchable knowledge map that keeps track of sites relevant to information systems managers. AT&T has a Business Network Bookmark (www.boet.att.com) with links to management resources.
In 1995 and 1996, the number of web sites doubled roughly every six months. While that growth rate will not continue forever, it is clear that the number of web sites is growing faster than most of us can keep up. By identifying key resources that track growth within our own disciplines, we can reduce the struggle to sort through thousands of hits in a search engine.
References
Finch, Byron. The Management Guide to Internet Resource, 1997 edition. McGraw Hill Companies, Inc. 1 997.
Martin,Justin. "Web Watch," Fortune, November 11,1996.