The following analysis and interpretation will be presented
according to the following subject question blocks used for the design
of the questionnaire (see blocks of questions and the
design of the questionnaire).
If one is only interested in the results of special fields, one can
use the following links:
-
The respondents of the questionnaire
(first block of questions PART
I, questions 1 to 3 and twelfth
block of questions personal
information)
-
The use of the Internet by the participants
and how they look for information (second question block
PART I, question 4)
-
The technical equipment of the participants
(third block of questions PART I, questions 5 to 6
and thirteenth block of questions technical
information)
-
The expectations for a one-stop-shopping
site (fourth block of questions PART
II, question 1)
-
The respondents' ranking of search
fields and preferred form of search and navigation (fifth question
block PART II,
questions 2-3)
-
The most interesting information
resources for the participants (sixth block of questions
PART II, questions 4-
6)
-
The most useful representation in
the view of respondents (seventh block of questions (PART II,
question 7)
-
The need for document delivery
(eighth block of questions PART
II, question 8):
-
The need for multi-lingual features
(ninth block of questions PART
II, questions 9 to 11)
-
The usage of search engines and
their evaluation by the respondents (tenth block of questions
PART II, questions 12
to 13)
-
Desired functionality of EULER
(eleventh block of questions PART
II, question 14)
-
Interest in the development of EULER
by the participants (twelfth block of questions
personal information)
The use of the Internet by the participants and
how their look for information
(second block of questions
PART I, question 4)
| general services on the net: |
frequency rank
(mean) |
relevance rank
(mean) |
| Internet services in general |
1 (1.22) |
1 (1.52) |
| WWW |
2 (1.28) |
2 (1.62) |
| Internet search engines |
3 (1.95) |
4 (2.04) |
| library online catalogues |
4 (2.46) |
5 (2.05) |
| bibliographic databases |
5 (2.63) |
3 (1.87) |
| other online access |
* (2.90) |
* (2.60) |
From this table, you can see that Internet services in general and
the WWW are often used by the participants daily, Internet search engines
and library online catalogues more weekly and bibliographic databases more
monthly. About 34% to 40% of the participants use Internet search engines,
library online catalogues, bibliographic databases and bibliographic/reference
databases on the Internet every week. The participants are very familiar
with the WWW; so the planned EULER user interface
by the WWW is a good choice from this point of view.
Other online access is not important for the group of all 132 participants.
Only 11% of them answered the question of frequency of other online access
with frequency at least ones a week. Only 13% of all 132 participants thought
of relevance of other online access for their work. But it can be seen
that for the people, who use other forms of online access, those forms
are quite relevant (more than 57%). The participants mentioned for other
online access: ftp, telnet, email, news, online databases and university
libraries (see details).
That means that more general services will
more often be used than special Internet services. This could be
expected. The results show that the participants have experience
with general electronic information systems on the Internet; that
is above the average. E.g. Rutenfranz mentioned as
one result of their empirical study that 64.3 % of persons belong to the
group of mathematicians and scientists use their computer daily, 30 % more
than ones week, 4.3 % one a week and 1.4% with a lower rate (see, Rutenfranz,
1997, p. 222). The results in this study are much higher. If you compare
the order of frequency and relevance you can see that Internet
search engines are used more often than evaluated relevant (see
Figure 6).
Figure
6: Frequency and relevance
of Internet search engines (frequency (light blue bars)
total answers: 127 (on 127 completed forms)
A) 1/day: 47 (37%), B) 2/week: 50 (39%), C) 3/month: 22 (17%), D) 4/less:
5 (4%), E) 5/never: 3 (2%); relevance (dark
blue bars) : total answers: 124 (on 124 completed forms) A) 1/highly
important: 34 (37%), B) 2/important: 41 (33%), C) 3/no opinion: 31 (25%),
D) 4/unimportant: 6 (5%), E) 5/not at all important: 4 (3%))
In the view of respondents bibliographic databases
are more relevant than they were used. For
the users' behaviour there could be different possibilities, e.g. the archived
results were seen as not very good (perhaps by search engines) or the usage
of information systems is too difficult. Another reason could be the access
of the information systems. In the next table the access of bibliographic
databases is analysed.
Further details:
57% of persons work at least once a week with library
online catalogues. 73% of participants chosen "1" or "2" for the relevance
of library online catalogues for their work. 52% of persons work at least
once a week with bibliographic databases. 74% of participants chose "1"
and "2" for the relevance of bibliographic databases for their work.
See all Figures in the detailed
annex.
Access of bibliographic and reference databases by:
| access |
frequency rank
(mean) |
relevance rank
(mean) |
| access by Internet |
1 (2.18) |
1 (1.62) |
| access by CD-ROM standalone |
2 (2.87) |
* (3.74) |
| access by CD-ROM network |
3 (3.83) |
* (3.48) |
| other online access |
* (4.32) |
* (3.20) |
Access of bibliographic databases by the Internet
is used rather weekly, access by CD-ROM standalone rather monthly and CD-ROM
network access is used in most cases less than ones a month. The
access by the Internet could be the most convenient way of access bibliographic
databases. The low ranking of local access to databases CD-ROM differs
from other studies
(in the study by the Sachverständigenkommission
(1995 p. 26) you could find the relevance for local databases with
60% for great relevance ("große Bedeutung"). ). Also in contrast
to the findings of the questionnaire for example in the University Göttingen
we have a huge CD-ROM Net- about 120 CD-ROMs - which are used by the university
and usage increased every semester.
Further details:
Over 50% of the participants never use CD-ROM network,
CD-ROM standalone and other online access for bibliographic/reference databases.
For 50% of participants the CD-ROM networks, CD-ROM standalone and other
online access were not relevant. Names of the
used CD-ROMs can be seen here.
See all Figures in the detailed
annex.
Usage of electronic full text resources:
| electronic full text resources |
frequency rank
(mean) |
relevance rank
(mean) |
| electronic preprints |
1 (2.98) |
1 (2.33) |
| electronic journals |
2 (3.18) |
2 (2.42) |
| electronic lecture notes/conference proceedings |
3 (3.46) |
3 (2.69) |
| electronic delivery systems for print-only material |
4 (3.91) |
5 (3.30) |
| electronic books |
5 (3.98) |
4 (3.19) |
| electronic dissertations |
6 (4.13) |
6 (3.32) |
| other full-text sources on the net |
* (3.91) |
* (3.53) |
The participants use the following
electronic resources in this order: first preprints, second journals, third
lecture notes/conference proceedings, fourth delivery systems, fifth books
and seventh dissertations. The average
respondent uses electronic preprints, journals and lecture notes/conference
proceedings at least once a month and electronic delivery systems,
books and dissertations less than once a month.
A possible reason for this behaviour could
be the actuality of mathematical knowledge in the different resources especially
the relevance for the participants mathematical research work. Resources
with high actuality were more often used by the participants. The
high ranking for electronic preprints shows that an electronic
library has a different centre of resources than a "normal library".
The frequency of usage is similar to the meaning
of relevance. Only for electronic delivery were lower evaluated and electronic
books higher. The results for electronic books
are astonishing because today there are a few electronic books available.
The questions which electronic books the participants use should be examined
further.
Further details:
Only 4% of persons work at least once a week with electronic
books, 31% at least once a month and 70% less or never. 33% of participants
chose "1" and "2" for the relevance of electronic books for their work
and 44% "4" and "5" for unimportant. It could be seen that the relevance
of using electronic books is divided.
For the values of the group of librarians see annex to
part I (click here). 27% of persons work
with electronic dissertations at least once a month and 73% less or never.
31% of participants chose "1" and "2" for the relevance of electronic dissertations
for their work and 45% wrote that they are unimportant with the answers
"4" and "5". It could be seen that the meaning to relevance and the frequency
of electronic dissertations of using electronic dissertation is very split.
67% of persons work with electronic preprints at least once a month and
32% less or never. 61% of participants chose "1" and "2" for the relevance
of electronic preprints for their work and 20% wrote that they are unimportant
with the answers "4" and "5". Electronic preprints are labelled to be very
relevant for the work and are used very differently by the participants.
The graphics for frequency and relevance do not correlate.
50% of the persons work with electronic lecture notes
/ conference proceedings at least once a month and 50% less or never. 50%
of participants chose "1" and "2" for the relevance of electronic lecture
notes / conference proceedings for their work and 32% wrote that they are
unimportant with the answers "4" and "5". This shows that the electronic
lecture notes / conference proceedings are used very different from their
assessment to be relevant.
30% of persons work with current electronic delivery
systems for print-only material at least once a month and 71% less or never.
37% of participants chose "1" and "2" for the relevance of electronic delivery
systems for print-only material for their work and 49% wrote that they
are unimportant with the answers "4" and "5". Electronic delivery systems
for print-only material was labelled from nearly the half group of participants
as unimportant and only a third use current electronic delivery systems
for print-only material regularly.
33% of persons work with other full-text sources on the
net at least once a month and 67% less or never. 27% of participants chose
"1" and "2" for the relevance of other full-text sources on the net for
their work and 57% wrote that these are unimportant with the answers "4"
and "5". It could be seen that more than half of the participants wrote
that other full-text sources on the net are not relevant. The frequency
of other full-text sources on the net is also low.
See all Figures in the detailed
annex.
The technical equipment of the participants
(third block of questions PART
I, questions 5 to 6 and thirteenth block of questions
technical information)
86% (113 persons with 3 multiple entries) of all participants of the
questionnaire are connected to the Internet by the Netscape browser and
at least 7% (9 persons with 1 multiple entry) by the Microsoft Internet
Explorer. 8% of the participants did not specify the browser they use.
The Internet Explorer is used almost equally in the versions 3 and 4. The
Netscape browser is most often used in version 3 and 4 (see details).
So it could be concluded that the participants followed the technical developments
of web services (e.g. Java applications) could be used by most of the participants.
But the technical standard for the EULER engine should
be Netscape in version 3 and the Internet Explorer in version 3.
Figure 6: Connection to the Internet
(Total answers: 173 (on 129 completed forms) A) campus: 116 answers, B)
business/corporate: 16, C) home: 41, D) other forms: 0 )
At least 88% of all 132 participants of the questionnaire
are connect to the Internet at their university office place.
The expectations for a one-stop-shopping site
(fourth block of questions
PART II, question 1)
| use of a one-stop information system |
personal opinion rank
(mean) |
| search items to get the document |
1 (1.49) |
| get an overview of the literature in some field |
2 (1.74) |
| keep up-to-date with developments in special areas |
3 (2.02) |
| get an overview of the publication activities of single authors |
4 (2.20) |
| other |
* (2.30) |
| search items for verification of citations |
5 (2.38) |
The most important use for our respondents is to reach items in the
information system in order to get the document,
the second one is to get an overview about literature in some field, the
third place there is the use to keep up-to-date with developments in special
areas, the fourth place is to get an overview about the publication activities
of a single author. And the last place is to search items for verification
of citations. This rating means that the EULER
system should provide a document delivery component by which the users
have the possibility to get the document. The second usage rating to get
an overview over the literature in some field and the third to keep
up-to-date with developments in special areas. The subject-oriented access
to EULER could satisfy this user needs.
Further details:
Using a one-stop information's system for getting the
document is for 91% at least important and for 63% very important. See
other usings of a one-stop-shopping site.
See all Figures in the detailed
annex.
The respondents' ranking of search fields and
preferred form of search and navigation
(fifth block of questions
PART II, questions 2-3)
The next Table tests the basic assumption that the user wants to search
in the EULER user interface the same categories as in common library online
catalogues.
| preferred search field |
personal opinion rank
(mean) |
| author |
1 (1.44) |
| keywords |
2 (1.72) |
| title |
3 (1.84) |
| other fields to be included |
* (2.29) |
| Mathematics Subject Classification (MSC) |
4 (2.48) |
| journal title |
5 (2.79) |
| other classification |
* (2.91) |
| type of publication |
6 (3.10) |
| date of publication |
7 (3.13) |
| publisher |
8 (4.02) |
The participants want to search
author (first), keywords (second) and title (third), followed by special
classification MSC and journal title. This
could be the possible order in the search interface.
It it very astonishing to see that the field 'date of publication'
and the field 'publisher' and have the lowest rates of all
fields, but we have to distinguish the different kinds of resources. The
higher appreciation of keywords compared to classification is astonishing,
as well. The result have to be further investigated.
Further details:
90% of the participants chose the category author as
their preferred search field and labelled this at least as important.
54% of the participants chose the Mathematics Subject Classification
(MSC) at least as important for their search. Only 25% chose very important.
83% of the participants chose the category keywords as their preferred
search field and labelled this at least as important. 51% chose very important.
29% of the participants chose the category date of publication
as their preferred search field and labelled this at least as important.
Only 6% chose very important. See
other classifcations. See other fields to
be included.
See all Figures in the detailed
annex.
form of search and navigation -
search in the database |
personal opinion rank
(mean) |
| menu-driven multi-field search |
1 (1.44) |
| features for menu-driven multi-field search: "wild cards" |
2 (1.63) |
| features for menu-driven multi-field search: selectable truncation |
3 (1.94) |
| features for menu-driven multi-field search: clickable boolean operators |
4 (1.98) |
| features for menu-driven multi-field search: proximity functions |
5 (2.52) |
| one line simple search in all fields |
6 (2.58) |
| command language interface |
7 (2.97) |
Further more they want a menu-driven
multi-field search/navigation with the following
features: first "wild cards", second "selectable
truncation", third clickable boolean operators and last proximity function.
It can be recommended from this questionnaire that the search and navigation
functions in EULER should be possible in a multi-field menu with the features:
"wild cards", selectable truncation and clickable boolean operators.
Further details:
92% of the participants chose the form of menu-driven
multi-field search as their preferred form of search and navigation and
labelled this at least as important. 68% chose very important.
See all Figures in the detailed
annex.
form of search and navigation -
browse through different database fields |
personal opinion rank
(mean) |
| index of author names |
1 (1.91) |
| keywords |
2 (2.01) |
| navigate through the classifications system tree |
3 (2.30) |
| title index |
4 (2.33) |
| other browsing fields |
* (3.44) |
The participants want to browse the author
names first, second keywords, third the classification tree and fourth
the title index. This is the participants' suggestion for the design
order of the browsing fields. It is astonishing that proximity functions
and command language interface are not rated very high. It is remarkable
that browsing keywords is ranked higher than browsing the classification
tree. A reason for this could be the common usage of library online catalogues
where classifications are not very often used by the users. Not all
mathematicians are aware of the difference between classification
and keywords. EULER could explain this in help text fields and should also
provide browsing by keywords. In recommendations "other browsing fields"
e.g. journal title and series names indexes
were given.
Further details: See
other browsing fields.
See all Figures in the detailed
annex.
The most interesting information resources for
the participants
(sixth block of questions
PART II, questions 4- 6)
| important information sources |
personal opinion rank
(mean) |
| bibliographic databases |
1 (1.42) |
| electronic journals |
2 (1.87) |
| preprints |
3 (1.91) |
| online library catalogues and databases (OPACs) |
4 (2.14) |
| proceedings |
5 (2.16) |
| other sources |
* (2.36) |
According to the participants of the questionnaire,
the most interesting information resources
for EULER are bibliographic databases (first rate), electronic journals
(second rate), preprints (third rate), online library catalogues and databases
(fourth rate) and proceedings. It could
be seen that the in the EULER project these information sources are represented.
Publisher catalogues, citations and address directories were suggested
by the participants.
Further details:
The respondents gave quite decisive answers about their
ranking "highly important" of information resources: 1. bibliographic databases
(84 answers), 2. preprints, 3. electronic journals (56), 4. OPAC library
catalogues and databases (50) and 5. proceedings (47). See
other sources.
See all Figures in the detailed
annex.
importance to restrict search
|
personal opinion rank
(mean) |
| restrict search on specific resource types |
1 (2.76) |
| restrict search on specific information providers |
2 (3.16) |
This table shows that here is no important need
to restrict search by resource types or specific information providers.
This is astonishing because the participants want
to get the document on the one-stop-shopping site.
Further details:
See all Figures in
the detailed annex.
The most useful representation in the view of
respondents
(seventh block of questions (PART II,
question 7)
| useful result representation |
personal opinion rank
(mean) |
| selectable long and short lists |
1 (1.90) |
| refine search |
2 (1.95) |
| selectable sorting by different fields |
3 (2.19) |
| automatic relevance ranking |
4 (2.70) |
The respondents of the questionnaire want to get a result
representation with selectable long and short list first,
then refine search and further selectable sorting by different fields.
It is remarkable that the participants do not want to get automatic relevance
ranking like they are used from common search engines.
Further details:
Regarding to the default or choices of result representation
the respondents have given the following ranking for "very important":
1. refine search; 2. selectable long and short result representation; 3.
selectable sorting of different fields; 4. sorting by automatic relevance
ranking.
See all Figures in the detailed
annex.
The order of the found item should be presented as
author, year, title and type; see Figure 8.
Figure
8: Preferred order of found items (total
answers: 203 (on 127 completed forms) A) by author: 90, B) by title: 36,
C) by year: 52, D) by type: 15, E) by other:
10)
It is astonishing that the participants evaluated the date as not so
important but they want to get the date in the result representation as
second item like in common form of citation in science.
Figure
9: Preferred length of display (Total
answers: 125 (on 125 completed forms) A) long: 33 (26%) ,B) short: 92 (74%)
)
The participants prefer short display lists (Figure
9) with 20 hits per page in the result
representation (Figure 10):
Figure
8: Preferred hits per page (Total
answers: 126 (on 126 completed forms), A) 10: 15 (11%), B) 20: 69 (55%),
C) 30: 21 (17%), D) 40: 4 (3%), E) 50: 10 (8%), F) more: 7 (6%) )
The need for document delivery
(eighth block of questions
PART II, question 8)
Figure
9: The need for document delivery (Total
answers: 120 (on 120 completed forms) A) 1/very important: 43 (36%),
B) 2/important: 28 (23%), C) 3/no opinion: 26 (22%), D) 4/unimportant:
19 (16%), E) 5/not very important: 4 (3%))
The answers to document delivery are divided. Automatic document
delivery was ranked high by a majority of the respondents (59%;
very important: 36% and important: 23%; mean value 2.28).
Figure
10: Getting the document (Total
answers: 125 (on 125 completed forms) A) scanned images (TIFF) by e-mail:
62 (50%), B) Ariel: 9 (7%), C) Fax: 7 (6%), D) copies via normal mail:
47 (38%))
EULER should provide a document delivery component via email or
postal mail by the document delivery services of the partners.
The need for multi-lingual features
(ninth block of questions
PART II, questions 9 to 11)
| importance of multi-lingual features for |
personal opinion rank
(mean) |
| user interface |
1 (3.71) |
| main menu |
* (2.94) |
| help texts |
* (3.03) |
| search interface |
* (3.15) |
| thesaurus |
* (3.16) |
| classification |
* (3.55) |
| others |
* (4.08) |
The multi-linguality in the user interface was evaluated very
low. This attitude to multi-lingual features could also be seen in the
low rate of received answers.
Further details (none representative
completion rate !):
The multi-lingual features are not very high ranked by
the respondents. Only 20% of them saw it as important or very important.
Of the 13 persons who gave the answer "very important"
six persons come from France and two persons come from Germany. Further
more one person from Mexico, one from Russia, one from Columbia and one
from Sweden answered that multi-lingual features are very important. From
the 26 people chose "very important" and "important" are 11 (42%)
from France, 4 (15%) from Germany, 2 (8%) from Italy, 2 (8%) from Sweden,
2 (8%) without any e-mail address and 1 (4%) from each of the following
countries: Mexico, Russia, Columbia, Italy and Switzerland.
65% of all French participants (17 persons) and 29% of all
German participants (14 persons) of the questionnaire underlined the importance
of multi-lingual features (13% of all 16 Swedish respondents, 13% of all
16 Italian participants). It could perhaps be conclude that the multi-lingual
feature should be beside the English version designed in French and perhaps
in German.
Respondents, who are in favour of multi-lingual features
("very important") prefer them in the main menu (18 persons), the help
texts (16 persons), the search interface (11 persons) and the thesaurus
(11 persons). See other desired multi-lingual
features.
See all Figures in the detailed
annex.
Figure
11: The choice of languages (Total
answers: 100 (on 51 completed forms none representative
completion rate) A) French: 34 (34%), B) Spanish: 10 (10%), C) Swedish:
5 (5%), D) Dutch: 6 (6%), E) German: 21 (21%), F) Portuguese: 3 (3%), G)
Italian: 15 (15%), H) others: 6 (6%) )
The favour for Italian and German is as high as the representation of
Italians and Germans/Austrians among the participants. More than twice
than the French representation is the favour for French. Swedish and Dutch
are requested less than their respective representation.
Further details:
Of 132 entries 71 have characteristics of languages
(64 English [en], 5 French [fr] and 2 German [2]). Of the 71 entries 90%
stand for English, 7% for French and 3% for German.
Figure
12: The choice of languages in the Internet
browsers (none
representative completion rate because of 46% all questionnaires have no
statements)
The usage of search engines and their evaluation
by the respondents
(tenth block of questions
PART II, questions 12 to 13)
There were several given reasons for the using of Alta Vista. The form
of the question allows no easy interpretation. Some answers were put together
under some points (problem of analysis):
| reasons for usage Alta Vista problem
of analysis |
ranking (possible answers for the usage of 37 answers) |
| number of (relevant) hits |
*1 (8) |
| easy usage |
*2 (6) |
| not very selective/ wide range |
*3 (4) |
| fast |
*4 (4) |
| familiar |
*5 (3) |
Further more reasons were mentioned, e.g. search for French sites, possibility
to refine the search or free selection of matches (see details).
reasons for usage MathSciNet/ Zentralblatt/MATH
problem of analysis |
ranking (possible answers for the usage of 37 answers) |
| little garbage/relevant information |
* (5) |
| access in the institution |
* (5) |
| fast |
* (3) |
| bibliographic information |
* (2) |
| wide ranging |
* (2) |
| reliable |
* (2) |
The reasons for usage of other search engines have very few statements
from the respondents. The usage of the different search engines has in
both cases the reason to get fast relevant information
with wide ranging. But from the statements of the participants we
could see two different views to get to get fast
relevant information with wide ranging. One view is close related with
a specialized subject-oriented access to the field of work.
The question what makes a search engine interface design effective
was answered only by a minority (only 29 persons filled out the text field).
In most cases simplicity was the answer for
Alta Vista and Zentralblatt/MATH. For details see here.
Desired functionality of EULER
(eleventh block of questions
PART II, question 14)
Here are some requests for the functionality of the EULER engine:
-
refine search:
"possibility to get lists of the database; possibility to refine a
search" "1. The subscription price should be kept to a minimum. 2. Refined
search capabilities on the output of the initial search are ESSENTIAL (a
serious drawback of MathSciNet is their absence). 3. For bibliographical
databases, it is important to have the information about translations,
if any (often missing in MathSciNet and Math Reviews)."
See the question
to refine search.
-
help texts:
"Good online help, especially on Boolean operators"
-
links:
"many links to other information provider/systems, with a short description
of what one would have to expect from the other provider/system."
-
taxonomy:
"kind of mathematical taxonomy tree preferably better organized than
AMS classification.",
-
display:
"navigation with hypertext links (authors, keywords)in the results
pages."
"I believe that it is convenient to have a possibility to select several
items from results of (possibly) several search procedures and to save
or print out them."
-
MathML Display:
"personnel electronic shelf "Please take into account the upcoming
MathML standard for display of mathematics via the web. I believe MathML
will greatly facilitate the display *and* content-cataloging of math on
the web. It will eventually allow us to read mathematics on the web directly
via browsers."
-
teaching:
"tree of categories, chains of information organised in something like
tutorials (e.g. "Everything you want to know about trigonometry)"
"Overviews of Textbooks in Mathematics for Undergraduates, in particular
new stuff. Listed by type of course e.g. "Basic Linear Algebra", "Mathematics
for Biochemistry", (or whatever)"
-
access to the EULER system:
"No, but show me how to start my searches from my own homepage! (See
Q13.)"
For details see here.
Some desired functionality for EULER are:
refine search, good help texts, many links, display selection for printing
and saving search results, usage of MathML, looking for teaching material,
special entrance points to the EULER search engine by personal homepages
and the geographic distribution of resources (none
representative completion rate).
Interest in the development of EULER by the participants
(twelfth block of questions
personal information)
The following Figure shows the high interest of the participants of
the EULER user questionnaire. The persons gave their email address to get
further information about the project.
Figure 13: Respondents' email address rate for further information
about the development of EULER
|