CHAPTER SEVEN
FRAMEWORK VALIDATION
7.1 Introduction
This chapter focuses on the evaluation and validation of the framework. The research validity and method deployed to maximise the validity will be discussed in the first part of this chapter. Also, the eternal framework validity will then be discussed. The second part of this chapter will analyse and discuss the results obtained from the evaluation of the framework. The framework evaluation has been carried out by academics and experts practitioners and has measured the suitability and effectiveness of the framework in the construction sector.
7.2 Validity
The validation process started at the first stage of the research, making sure that the research participants, measuring instruments and context were exclusively accomplishing the research aim. A number of writers have contended that validity cannot be applied to qualitative research. At the same time, have understood the need for some qualifying check or measure for their research (Golafshani 2003). Clearly there are no direct tests validating qualitative research; however, this does not imply that there are no guidelines (Patton 1990). Through the stages of this research, the effort has been made to guarantee that: (1) the measuring instrument is measuring the concept being referred to and no other concepts; and (2) there is an accurate measurement of the concept (Bailey 1994). The measuring instruments used in the interview method were the interview questions; also the method used in analysing the data collected and building the framework. Therefore, it was important to ensure that all research propositions were clear and effectively measured. Also, the method of analysis required the provision of right inferences.
Get Help With Your Essay
If you need assistance with writing your essay, our professional essay writing service is here to help!
Essay Writing Service
Qualitative research has often been portrayed by flexibility and uncertainty, and tension between creativity and rigour. Therefore at the interview analysis and the stages of framework building, a critical compromise was carefully considered (Pyett 2003). The analysis of interviews engaged “a deductive process” whereby the author was “his most trenchant critic” (Cook and Campbell 1976, p: 229). This is known as internal validity. However, in this study, an open viewpoint of validity was used whereby different techniques enhanced the analysis and understanding of the practice of knowledge capture and retrieval and prompted more reliable and valid development of the framework (Golafshani 2003). These methods included:
An in-depth revision of the instruments used for data collection and the method of analysis
Carrying out a peer review of interviews themes and methods of analysis
Carrying out a pilot study; assessing the language of the questions; definitions and terms; clarity of the purpose of questions for interview
Construct a descriptive and relational framework for the interview analysis to make sure there is the consistency of analysis criteria and accurate interpretation and inferences.
7.3 External Validity
Developing a framework for the capture and retrieval of WLC knowledge in a construction project is based on a small number of cases, and does not mean this framework can be suitable and effective to other construction organisations practising WLC in the construction sector. However, researchers are also interested in the generalisation of the findings beyond the investigated cases (Schwab 1999). The justification for carrying out an in-depth interview is that people involved in a matured practice have knowledge that would somehow or another not be accessible to the researcher. Therefore, it is the quality of the understanding that is vital, instead of the number of respondents who share it (Wainwright 1997).
While researching a few cases may accomplish excellent validity by giving a significant understanding of the practice in those cases; they have been broadly criticised as lacking external validity (De Vaus 2001). The generalising of findings to or across a target population sample is known as external validity (Pedhazur and Schmelkin 1991). There are no efficient or verifiable methods to carry out external validation based solely on a single examination of a research relationship (Schwab 1999). However, a definitive way to assess the validity of the proposed framework is to test such principles in reality, yet within research practice, this is rarely conceivable (Pyett 2003).
Framework testing by the application method, it could take quite a while before the reasonable judgment of the framework validity could be reached. Regardless of the possibility that this was to take a short time, there would be a need to access to a huge sample (representing the industry) and the framework implementation in this sector. However, in this particular case, this was viewed difficult. Nevertheless, a few other methods were able to amplify the framework validity in this study as outlined:
Select example case studies organisations (of current and best practices) from the target population in the interview stage.
The pilot study provided a solid ground for understanding the current practice of KM; and therefore it was utilised as guiding principles in using the best practice to suit the industry, which boosted the generalising or external framework validity.
Looking for the evaluations of the suitability and effectiveness of the framework in the construction sector by experts practitioners and academics.
7.4 Framework evaluation
Testing the general framework feasibility by applying it to a large number of organisations representing the construction sector have been difficult in this research. The framework evaluation by experts was viewed as another alternative for evaluation. in fact, this method was deemed as appropriate with regards to risk, finance and value.
Assuming that 100 randomly selected organisations are a representative sample and the resources required are accessible in those organisations, it would take six months for the testing and implementation of the framework (practically it would take years). However what if the framework did not work? This would mean lost time of 600 months (100 organisation X 6 months) and therefore significant effort. Therefore, in this case, the evaluation method by experts could be deemed the most appropriate method even though the “generalise by implement” method was deemed possible.
The reason for the evaluation method accordingly was not to anticipate that the framework was suitable for the organisation of the participated experts; to obtain the expert’s evaluation in regards to the frameworks suitability in the industry. The suitability of the framework was not seen adequate however because the suitable framework did not necessary means it would add value. Therefore, effectiveness was another criterion the evaluation sought to answer.
7.4.1 Participants
Based on their experience, the suitability and the effectiveness of the framework can be judged by experts by using the right tool techniques, required resources and methods. The time and effort required for the framework implementation can also be estimated and matched with the current practice.
The participants involved in the framework evaluation is made up of two groups of experts: academic and practitioners. The practitioners are involved with WLC practice in the construction project and therefore have a better understanding of the daily work. Also, their participation in a project which requires WLC practice means they can work with other organisation who practice WLC in the construction sector and can understand the verity of work environments.
Also, academics are expected to have a deep understanding of different methodological viewpoints, as they know about observation and analysis. Further, the academics rely greatly on the construction industry as a major source of information, so does not imply that they will be isolated. Also, many academics are involved in the practice.
A total number of sixteen experts were divided equally between the academics and the practitioner who participated in the framework evaluation. The participants year of experience ranges from eight to thirty-eight years, accounting for the total of three hundred and thirteen years of experience. The eight practitioners were from the three case study organisations who participated in the pilot and main interview of the study. The focus on practitioners was on those in the field with experience in WLC practice because they had an in-depth understanding of the WLC practice. Also, participating academics were from four universities in the UK, working in the built environment and information and knowledge management departments.
Experts
Job title
Year of experience
Organisation A
Project Manager
35
Quantity Surveyor
20
Building Surveyor
7
Organisation B
Quantity Surveyor
6
Project Manager
33
Organisation C
Cost Estimator
15
Project Manager
27
Construction Manager
7
Senior Lecturer
12
Senior Lecturer
30
Lecturer
6
Lecturer
7
Lecturer
25
Lecturer
25
Reader
31
Reader
21
Table 7.1: Characteristics of participating experts in the framework evaluation
Because the framework evaluation required a seminar presentation, this process was thought to be long, and it was expected a low rate of response. Therefore, it was decided to reach the target of participants by sending invites to a member of staff with the three case study organisations. The total number invitations forwarded was 300 of which 120 were sent to academics; and 180 to the three case study organisations.
7.4.2 Method of Evaluation
Emails was used a tool to invite experts and included;
An overview of the research project
Request to participate in the evaluation of the framework for the purpose of validity
Request on the best times which suits each of the case study organisation for a 40mins seminar presentation
The framework was presented to each case study organisation using Microsoft Office PowerPoint 2007. The presentation was interactive in which participants lessened the explanation; while diagram parts were moved in each presentation slides accordingly. The seminar presentation was arranged into ten slides which took a narrative sequence of logic since it began by clarifying the parts of the framework independently and afterwards presented the strategic framework.
Figure 7.1: A screenshot of the PowerPoint presentation
The questionnaire was designed and hosted using the online service provider: “Survey Monkey’ and consisted of two parts. Firstly, details about participants including name, organisation, job title, business and size of the organisation.
secondly, the framework evaluation and this was included in two sections
Score based questions to the rate the suitability and effectiveness of the developed framework
Seeking the opinions of the participants about the advantages, limitations and ways of improving the framework.
It was decided to present the framework diagrammatically showing the strategic, tactical and operational levels. The presentation gave a more in-depth description; however diagrams were sent to four research students to assess their readability, and the outcomes affirmed that such diagrams could convey the concept and details of the framework if the reader had extensive experience in construction management. One of the research students who took part in the pilot study was a medical doctor, dissociated from the construction industry, who found the framework unreadable, Although he has a good understanding of the major concept. Respondents were also sent an attachment of the presentations slides via email.
7.5 Results
7.5.1 Suitability of framework
Participants were being asked to show their rating for suitability for each of the components of the framework; where 1 implied unsuitable and 5 suitable. The answer from the analysis uncovered that the levels of suitability for all approaches were considered far above the ground.
As shown in Table 7.2, 3.74 out of 5 is the lowest mean value for the architecture of the knowledge base. Although it is not indicated in the value that the architecture of knowledge base is a 100% suitable, rather, it shows that it is likely going to be more suitable than unsuitable. The circumstance with regards to knowledge capture, retrieval and structure approach much better as they all have a score of 4 out of 5. The skewness measure has a negative value which shows that the mass of distribution concentrates on the side of suitable.
Knowledge Capture
Knowledge retrieval
Knowledge Structure
Architecture of Knowledge Base
N
Valid
16
16
16
16
Missing
0
0
0
0
Mean
3.9375
4.0625
4.0000
3.7500
Std. Deviation
.85391
.85391
.89443
1.00000
Skewness
-.605
-.863
-.639
-.343
Table 7.2: Suitability of the framework components
Figure 7.2 shows that the majority of the participants gave a 4 or 5 rating of the suitability of all the framework components. With regards to the architecture of the knowledge base, the suitability was rated under 3 by two participants (i.e. 2). The normal curve illustrates that the approximation of the average of all components almost falls in the 4 rate of suitability. All the indications provide evidence that those components are suitable for the construction organisations.
7.5.2 Effectiveness of framework
Participants were being asked to show their rating for effectiveness for each of the components of the framework; where 1 implied to be ineffective and 5 effective. The mean value of effectiveness for all components as shown is (Table 7.3) are above 3.75 and ranges between 3.87 and 4.18. The skewness measure has a negative value which shows that the mass of the distribution is concentrated on the side of effectiveness.
Knowledge Capture
Knowledge Retrieval
Knowledge Structure
Architecture of Knowledge Base
N
Valid
16
16
15
16
Missing
0
0
1
0
Mean
4.0625
4.18750
3.9333
3.8750
Std. Deviation
.85391
.910586
.79881
1.02470
Skewness
-.863
-1.019
-.842
-.571
Table 7.3: Effectiveness of the framework components
As shown on the histograms shown in Figure 7.3 shows that the majority of experts participants evaluated all components to a score 4 or 5 rates of effectiveness. Knowledge retrieval thus turns out slightly more effective than the other framework components. The normal curve makes it clear the approximation of the average rate of all components which almost lies in the 4 rate of effectiveness. All the indicators together provide evidence that those components will be effective when implemented.
7.5.3 The Entire Framework Suitability
Experts participants were asked to indicate their rate of suitability for the entire framework using the same criteria as mentioned above. As shown in Table 7.4, the mean value of suitability (3.93) and the negative value of the skewness measure provide an encouraging indication for the external validity. In Figure 7.5, the pie chart shows that the majority of experts participants (62%) rate the suitability 4 out of 5; and the vast majority (81.3%) are either giving the framework a rate of 5 or 4. Furthermore, the histogram in Figure 7.4 shows that only one expert gave a rating of suitability under 3 (i.e. 2). Therefore, the indications of suitability are positive and show that this framework is considered suitable.
The Entire Framework Suitability
N
Valid
16
Missing
0
Mean
3.9375
Std. Deviation
.77190
Skewness
-.881
Table 7.4: The entire framework suitability
7.5.4 Effectiveness of the Entire Framework
Expert participants were asked to indicate their rate of effectiveness for the entire framework using the same criteria as mentioned above. The mean value (4) and the skewness measure value (negative) which shows that the framework is evaluated to be effective as shown in Table 7.5. A third of the expert’s participant shows a full mark of effectiveness, and 40% scored the framework a rate of 4 for effectiveness as shown in Figure 7.7. However, as shown in Figure 7.6, three experts rate it a 3 and one a 2. Nonetheless, the normal curve is pointing to an approximation of effectiveness at the rate of 4. Indications show that the framework is effective.
Effectiveness of the Entire Framework
N
Valid
15
Missing
1
Mean
4.0000
Std. Deviation
.92582
Skewness
-.623
Table 7.5: Effectiveness of the entire framework.
7.5.5 Experts Opinion about the Framework
The expert participants were requested to express their opinion about the framework regarding suitability, effectiveness, advantages and limitations. This could give additional assessment not covered in the closed-ended questions. Twelve of the feedbacks were received and had been listed below:
It looks good to me. The problem is getting it work in practice.
Great model and l like it, particularly the knowledge forms and the categorisation. This will help save time when searching for knowledge
I think the overall seminar presentation was good and it covered a lot of the things we do not consider as worth pointing out in our day to day project delivery implementation.
It is very hard to make a judgment on these key issues just on the presentation. However, I am quite inspired by the approach to knowledge capture. Although the presentation is great, however, the information, it provides is insufficient.
It seems very general.
The framework works well in suitability and effectiveness; and in use of collaboration tools, intranet tools with knowledge management within organisations. It is essential for a large organisation that staff can find and utilise answers as well as gaining access to the subject matter, experts and sharing information on similar projects and programmes. The advantages are that knowledge is shared rather than lost or protected by individuals. The saying that knowledge is power is true of individuals; However in organisations that are able and willing to capture knowledge and share this with the wider team rather than isolating it with individual project members.
The framework is suitable and effective in many areas and an insightful and logical structure. There is some duplication, but that would seem to be unavoidable.
Leadership support is important, with top-bottom commitment. Co-operation/sharing is not always a “natural” inclination in individuals.
The framework seems to be highly advanced and would require an IT system to enhance its processes and to encourage people to use it. If all members of the organisation are contributing positively to the framework; and most of them have the ability to capture, retrieve, and then re-use the captured knowledge, then the system will be for sure highly effective. In reality, I think we share the knowledge “sometimes”, but even when we do so, we do not document or capture them properly. As a result, we lose whatever we have shared as we cannot capture, structure and re-use all the knowledge captured by only relying on our brain capabilities. We as practitioners need such a framework to assist us to have a better understanding of the knowledge so we can use it appropriately. From an overall point of view, I admire you for this excellent presentation, and it was informative and well structured. We have learned a lot from this insightful presentation about knowledge management. There is no doubt that this framework is needed for business success.
Although the framework appears to be complicated on its operational side; I am certain that if the KM operation depends on this model, the result will be relevant and highly efficient.
The framework makes sense from a project management perspective. However, knowledge capture and retrieval are complex and resource intensive activities and can be onerous to the project team if proper incentives are not aligned with the leadership’s expectation of knowledge sharing.
I think if your framework is properly applied, it will be extremely effective. Also, it is highly suitable. Regarding its advantages, I think it is simple to read and understand by any person who has a basic level of knowledge management and its management.
The issues mentioned in the comments can be grouped into four categories: Firstly, those which highlight the role of knowledge capture and retrieval.
Secondly, those who stress the importance and need for such a framework, due to the awareness of the problem, and thus confirm the motives for this research.
Thirdly, indications that the framework may require a huge effort and construction projects already involved in demanding work. This issue has been discussed in the statement of the problem (section 1.4) whereby the problem lies on the capture and not the retrieval side. To minimise the problem, the project review was adopted as the main technique for knowledge capture which already exists in projects, rather than introducing a new approach. Additionally, the daily based reviews were rejected in this research for the same reason. Furthermore, a strategy for motivation has been pinpointed in the framework to ensure the application of knowledge capture.
Fourthly and most importantly, a suitable and effective framework will also require careful application and effort. This has been said by several experts, and therefore considered a critical point, because the good theory does not necessary guarantee successful application.
Finally, an expert described the framework to be very general. As aIn fact the framework is considering a particular area to investigate, related to knowledge capture and retrieval of project management processes, in the context of the project. Models inside the framework have provided precise details regarding what to do in particular scenarios. Also, knowledge capture and retrieval cannot be tackled in isolation from issues such as.
Leadership support and cooperative culture. It is worth mentioning that this expert selected the rating of 2 for all of the 10 rating-based questions which are considered statistically biased. However, it was thought that this academic expert provided his personal details and these matched the tracking information. Therefore it was considered a valid response, and his opinion was appreciated.
Essay Writing Service Features
Our Experience
No matter how complex your assignment is, we can find the right professional for your specific task. Contact Essay is an essay writing company that hires only the smartest minds to help you with your projects. Our expertise allows us to provide students with high-quality academic writing, editing & proofreading services.
Free Features
Free revision policy
$10Free bibliography & reference
$8Free title page
$8Free formatting
$8How Our Essay Writing Service Works
First, you will need to complete an order form. It's not difficult but, in case there is anything you find not to be clear, you may always call us so that we can guide you through it. On the order form, you will need to include some basic information concerning your order: subject, topic, number of pages, etc. We also encourage our clients to upload any relevant information or sources that will help.
Complete the order form
Once we have all the information and instructions that we need, we select the most suitable writer for your assignment. While everything seems to be clear, the writer, who has complete knowledge of the subject, may need clarification from you. It is at that point that you would receive a call or email from us.
Writer’s assignment
As soon as the writer has finished, it will be delivered both to the website and to your email address so that you will not miss it. If your deadline is close at hand, we will place a call to you to make sure that you receive the paper on time.
Completing the order and download