Open Collections

Empirical comparisons of system analysis modeling techniques

University of British Columbia

The development of information systems consumes an increasing share of economic resources. Over a trillion dollars worldwide is invested in information technology annually, and this investment is growing over $100 billion a year. This investment occurs despite failure rates for large information system development projects that are estimated as high as 75%. The large investment and high failure rates combine to create the potential for significant impact from information system development practices that are able to address these failure rates. Researchers, over the past thirty years, have studied factors that drive these high failure rates. One of the factors repeatedly mentioned in practitioner surveys is the importance of accurate communication in the "upstream" analysis and planning stage of a project. System development professionals are aided in their upstream planning through the use of information system development methods (ISDM's). ISDM's are modeling tools and techniques that are capable of representing information about an information system. Many alternative system analysis modeling techniques have been developed, yet few empirical comparisons of the alternative techniques have been completed. The lack of comparative empirical data has contributed to a proliferation of modeling methods and increased the confusion surrounding the adoption of system analysis methods by system development professionals and teachers. This study addresses the issue of empirical comparison of system analysis modeling techniques. A new instrument and empirical method is proposed for developing a comparison of the level of "understanding" that a participant is able to create by viewing a description of a particular domain. The level of "understanding" is addressed using three measures: comprehension, problem solving, and text reconstruction. The new measures of "problem solving", suggested by Mayer in the field of Education Psychology, and "text reconstruction" or "Cloze", suggested by Taylor in the field of Communications, extend empirical instruments previously used by system analysis researchers. To test the efficacy of the proposed instrument and method, two empirical studies were developed in this thesis. The first study used the new instrument to compare three development methods "grammars: Text descriptions; Structured Analysis (using Data Flow Diagrams and Entity Relationship Diagrams); and Object Oriented Diagrams. The study was labeled an "Intergrammar" comparison, as three grammars representing three fundamental approaches to developing an analysis model were compared. Two propositions, in regards to the intergrammar study, were tested. The first suggested that viewing descriptions created with diagrams would lead to a higher level of understanding than viewing a description based solely on text. This hypothesis was confirmed. The second hypothesis suggested that viewing a domain description created using an object oriented grammar would lead to a higher level of "understanding" than viewing a description created using the "Structured Analysis" approach. The results confirmed the hypothesis that the group of participants using the Object-Oriented grammar scored higher in "understanding" than participants using the Structured Analysis grammar. A follow-up protocol analysis was undertaken to illuminate why the participants using object methods scored. The analysis of these protocols indicated two things. First, participants using Structured Analysis made little use of the Entity Relationship Diagram (ERD). Second, participants seemed to favor the "object" concept when answering questions. These findings provide some empirical evidence that objects may be more "natural" cognitive constructs than those used in Structured Analysis. The second study revisited a study Bodart and Weber's study regarding alternative grammars for the Entity Relationship Diagram. A grammar using mandatory attributes and relationships with sub types, the other using optional attributes and relationships, were compared. The grammars shared a common primary grammar, therefore, the second study was labeled an "Intragrammar" comparison. The new instrument was again used in this study. The ontological constructs proposed in the Bunge-Wand-Weber (BWW) model were used to suggest the theoretical advantage of the grammar using mandatory attributes and relationships with subtypes. The results supported the theoretical advantage associated with mandatory attributes and relationships with subtypes. This intragammar study provided further evidence of the utility of the empirical instrument proposed in this thesis. This study has implications for future empirical research in system analysis. The empirical instrument described in this thesis extends previous empirical research instruments with the introduction of the problem solving and the Cloze task. In two studies, the new instrument has displayed the sensitivity to differentiate between treatment groups. The results from the two empirical studies suggest that object-oriented analysis may hold advantages over traditional structured analysis, and that mandatory attributes and relationships may be preferred to optional attributes and relationships in the entity relationship grammar.

Thesis/Dissertation

application/pdf

2009-07-20

For non-commercial purposes only, such as research, private study and education. Additional conditions apply, see Terms of Use https://open.library.ubc.ca/terms_of_use.

http://hdl.handle.net/2429/10939

Business Administration - Management Information Systems

University of British Columbia

UBCV

DSpace