Evaluating Research Design

1. What is the article about? Summarize the article in two to three paragraphs.
   This study uses text mining technology to examine and analyze flight passengers’
   sentiments, emotions, feelings, and reactions upon cancellation, rescheduling, or delay of their
   flights. The rapid advancement of information technology and new media has made it possible to
   communicate and express their emotions and reactions via public social platforms and social
   media networks. Social media has become a significant data source of researchers and
   enterprises leveraging novel data mining techniques to analyze trends and make predictions in
   the data analytics realm.
   This study retrieves 24,165 messages and reviews from SKYTRAX posted by clients
   between Sep 2011 and Mar 2019. Of the 24,165, about 5,700 expressed flight delays.
   SKYTRAX is a global website that collects and allows users to rate their airline experiences,
   including the quality of services they received. Travelers can locate the corresponding airport or
   airline on the firm’s website and then publish their personal experiences by evaluating their
   services. The researchers then employ “sentiment analysis” to categorize user comments and
   “co-occurrence analysis” to examine travelers’ fears and concerns on several elements of the
   flight service. Specifically, the researchers use “Lexicon-based text sentiment” categorization
   tools to classify to classify flight status. The results also revealed an increase in travelers’ focus
   on the “service” elements following flight delays. After delays, passengers expressed dramatic
   dissatisfaction with the airline and airport services (Song & Zhuang, 2020).
2. What is the research problem statement? To what extent does the problem
   statement communicate the research problem to the reader? What improvements, if

EVALUATING RESEARCH DESIGN 3
any, do you recommend for the problem statement? This section will be about two
to three paragraphs.
This article has not expressed the research problem clearly. However, the most probable
research problem statement would be “a lack of appropriate data mining tools to analyze
passenger’s sentiments and concerns from information collected public social platforms and
social media networks after flight delays.” It is clear that social media and other public social
networks, including blogs, have become new platforms that customers can express their opinions
and feelings about experiences with specific brands, products, and services. The only problem is
the lack of appropriate technology-based tools to extract and analyze this data into meaningful
information and trends (Song & Zhuang, 2020).
Personally, the problem statement does not communicate the research problem to the
reader. I recommend that the researchers first express an existing gap in knowledge in the subject
and their impacts – which calls for further research. For example, they should describe the flaws
in the current techniques used to examine passengers’ sentiments and opinions in the air industry
after flight delays, and their subsequent impacts. These “problems” should lay the foundation to
justify the researchers’ choice of new data classification and analyses tools.

3. What is the intent of the research? To what extent does the purpose of the study
   communicate what the researcher will do about the problem? Do any elements of
   bias exist in the researcher’s intent? What improvements, if any, do you recommend
   for the research? This section will be about two to three paragraphs.
   The research intends to use text mining technology to efficiently and automatically
   access, classify, and analyze flight delay text messages posted by customers on SKYTRAX after
   their flights have been canceled or delayed. SKYTRAX is a global website that collects and

EVALUATING RESEARCH DESIGN 4
allows users to rate their airline experiences, including the quality of services they received. The
study applies a lexicon-based “sentiment” assessment model to analyze texts and “co-
occurrence” analysis to determine customer’s apprehensions and concerns on various aspects of
the airline services following flight delays. Undoubtedly, the study’s intent communicates what
the researchers will do about the problem – classify and analyze flight delay text messages.
The only bias in the researchers’ intent stems from their preconceived beliefs and
expectations about passengers’ potential emotions and sentiments after their flights have been
delayed or rescheduled. Automatically, the fact that flight delays, cancellations, or reschedules
are “negative experiences” means that the researcher already has a predetermined undesirable
outcome they expect. Addressing this bias will require analyzing all passengers: those that
experienced delays and those that did not experience any delays. This can give a true reflection
of the quality of services of the airlines or airports and how clients feel about them, including
food, facilities, seat allocation, and crew behavior (Song & Zhuang, 2020).

4. What are the research questions or hypotheses? Does answering the research
   question or conclusions on the “hypotheses” resolve the research problem? This
   section will be about one to two paragraphs.
   The study has no actual research questions or hypotheses. However, the first potential
   research question would be: “what are the sentiments and emotions of passengers upon delay of
   their flights?” The second research question would be: “Do passengers’ attention to the service
   aspects of airlines and airports increase or decrease after their flights are delayed?” Therefore,
   the two hypotheses from these research questions would likely be: “passengers will more likely
   express negative emotions and sentiments upon delay of their flights” and “passengers’ attention

EVALUATING RESEARCH DESIGN 5
to the service aspects of airlines and airports will increase after their flights are delayed” (Song &
Zhuang, 2020).
Answering these potential research questions or conclusions on the hypotheses can
unquestionably resolve the research problem. For example, answers to the first question or
hypothesis will give information about the emotions and sentiments of passengers after their
flights are delayed – whether negative or positive. Responses to the second question or
hypothesis will likely determine whether the passengers’ attention and satisfaction will increase
or decrease.

5. Describe the quality, depth, and organization of the literature. What improvements,
   if any, do you recommend for the literature review? This section will be about one to
   two paragraphs.
   The literature review’s quality, depth, and organization are top-notch. The review
   demonstrates an in-depth synthesis of past scientific research and evidence. It highlights flaws in
   current literature about methodologies used in collecting and analyzing data about flight delays
   from social platforms and websites. For example, the researchers argue that previous work has
   only focused on examining negative, neutral, and positive texts. The disadvantage is that these
   kinds of analyses make it challenging to define the “complex inner world of human beings based
   on two- or three-class sentiment analysis” (Song & Zhuang, 2020, p. 2). The literature review
   also details the practical and successful application of “sentiment classification” tools in other
   industries, such as tourism. The literature review is organized into two themes: “text sentiment”
   and “public opinion about flight delay” analyses.
6. What is the research design? Is the research design correct for the study’s intent
   and type of data used in the study? If so, why? If not, what research design and data

EVALUATING RESEARCH DESIGN 6
analysis do you recommend? To what extent do the data collection and analysis plan
gather and assess the data to answer the research questions or hypotheses? This
section will be about two to three paragraphs.
The research uses a cross-sectional, phenomenological-case study approach to examine
text messages by passengers after their flights are delayed. A phenomenological approach is
correct or appropriate for the intent of the qualitative study and data used because it allows the
researchers to gather and analyze the conscious experiences, sentiments, and emotions of
passengers after their flights are rescheduled or canceled. Of the 24,165 messages and reviews
retrieved from SKYTRAX, 5,700 were associated with flight delays. Phenomenology analyzes
phenomena linked with conscious human experiences, such as actions, perceptions, judgment,
emotions, and feelings. The study can be described as a “case study” because it investigates
various aspects of a specific case: “passengers that experience flight delays.”
The researchers’ data collection and analysis plan undoubtedly gathers and assesses
correct and sufficient data to answer the research question and hypotheses extensively,
effectively, successfully, and satisfactorily. A total of 5,700 flight delay texts from SKYTRAX is
adequate to make logical analyses and inferences. The four sentiment analysis technologies
employed are also relevant in answering the research questions and hypotheses, including
Lexicon-based text classification, document-level sentiment analysis, sentence-level sentiment
analysis, and co-occurrence analysis. Lexicon-based text classification using Vader and Pattern
model gives four output values: compound, neutral, negative, and positive scores (Song &
Zhuang, 2020).

EVALUATING RESEARCH DESIGN 7

7. Are the results, conclusions, and recommendations logical and reasonable based on
   the data analysis outcomes? If so, why? If not, what corrections do you suggest?
   This section will be about two to three paragraphs.
   The outcomes, conclusion, and recommendations are reasonable and logical based on the
   results of the data analysis. The results showed that flight delays could negatively impact
   traveler’s sentiments, views, or feelings. Passengers subjected to flight delays exhibit more
   negative emotions compared to those that experience positive emotions. The study associates
   these negative emotions to flight delays and the approach airlines employ to compensate
   passengers after their flights have been rescheduled or canceled. Additionally, the results
   revealed an increase in travelers’ focus on the “service” elements following flight delays.
   Passengers expressed dramatic dissatisfaction with the airline and airport services after delays.
   The study recommends airlines ensure flights are punctual to improve customer satisfaction and
   experience.
   Several reasons explain why the study outcomes, conclusions, and recommendations are
   practical and sensible. Firstly, evidence from previous research in several people-based
   industries has linked poor customer services, such as delays and low food quality, to negative
   customer sentiments, lower satisfaction, and customer loyalty (Supriyanto et al., 2021). Another
   reason is that conclusion and recommendations are based on interpretation of the study results,
   which point to the need to reduce delays and improve other customer-service elements to boost
   passenger satisfaction.
8. Describe the quality of the format and style of writing. What improvements would
   you suggest to the author to improve the document? This section will be about two
   to three paragraphs.

EVALUATING RESEARCH DESIGN 8
If I have to rate the quality of the writing and format style, a six out of ten would be
appropriate. The abstract does not give the reader a snapshot of the study arrangement; there is
no apparent research problem, research questions, hypotheses, and study design; and it does not
separate the data analysis methods and the results. Also, the study fails to use inferential
statistical methods like ANOVA and t-test to measure the validity and reliability of the test
results. Finally, there is no discussion section to conduct an in-depth examination of the
outcomes.
I would suggest that the researchers provide a clear and detailed abstract that reflects the
purpose of the study, study design, data collection and analysis, results, and recommendations or
conclusions. I would also recommend that Song & Zhuang clearly define the research problem,
research questions, and hypotheses. More so, I would suggest that the two separate the paper into
distinct sections, separating data analysis, results, and discussion sections. Finally, I would
recommend that the study uses descriptive and inferential methods to present data for easy
analysis and interpretation.

EVALUATING RESEARCH DESIGN 9

References

Song, C., Guo, J., & Zhuang, J. (2020). Analyzing passengers’ emotions following flight
delays- a 2011–2019 case study on SKYTRAX comments. Journal of Air Transport
Management, 89, 7. https://doi.org/10.1016/j.jairtraman.2020.101903
Supriyanto, A., et al. (2021). Effects of service quality and customer satisfaction on the
loyalty of bank customers. Cogent Business & Management, 8(1).
https://doi.org/10.1080/23311975.2021.1937847