Today we are going to analyze the article “Factors contributing to the mental wellbeing of Afghan migrants in Iran during the COVID-19 pandemic” authored by Khozaei et al. (2024) and published in the “Journal of Migration and Health”. This journal is a publication of Elsevier/ScienceDirect. The journal is indexed/abstracted in PubMed Central, Directory of Open Access Journals (DOAJ), Scopus (Q1), Emerging Sources Citation Index (ESCI) - Clarivate Impact Factor 2022: 4.60. This journal is publishing erroneous research without exercising good quality control practices which simply means that the Editors of this journal don’t read submitted papers while reviewers are not competent enough to identify major grammatical, theoretical, and methodological mistakes. This journal has been included on our list of “Questionable Journals”.

Let’s start reading and analyzing this article!

The Abstract starts with a grammatically incorrect sentence as highlighted in the picture below. Unfortunately, this is not the only mistake we highlighted but the article is full of language errors. It is very clear that neither the authors submitted their article for language editing nor the publishing journal provided any copyediting/proofreading services. Of course, we can’t identify every mistake but have highlighted only a few errors Editors of the journal failed to detect and rectify.

Once you read the “Introduction” of the article, you find that Research Question(s) the study endeavored to address was(were) not formulated and presented. In any research, Research Question(s) are considered as the backbone. The RQ(s) directs the researchers to choose appropriate Research Design. If there are no RQ(s), whole research becomes enigmatic.

In addition to the absence of RQ(s), no theoretical gap had been identified, at least at preliminary level, in the Introduction section. As researchers, we understand that RQ(s) actually reflect(s) the problem needing immediate attention. The problem can be theoretical or practical, but for academic research, the problem should be identified with theoretical support, however, we can’t find anything called “theory” in the “theoretical framework” of this paper.

All study hypotheses were not underpinned by any theory. It’s strange! First, the due care in formulating mediation and moderation was not taken. The linkages between IVs and DVs, the mediation, and moderation mechanisms were not established by conducting a thorough literature review. As researchers, we understand that mediational mechanisms are causal in nature and should be proposed following theory and logic. We know that variables like anxiety, stress, depression, and positive mental health lie under the domains of psychology and mental health. Why are we stressing on the points like lack of theoretical support and literature review? We try to explain here:

The study proposed that Anxiety causes Stress (see mediation hypothesis H2 on p. 3). As we know that both Anxiety and Stress are emotional responses but the main point to be considered is, Stress is caused by an external trigger (short-term or long-term/not persistent), whereas anxiety’s cause is internal (persistent). According to Healthline, Stress is “any demand placed on your brain or physical body”, whereas “Anxiety is a feeling of fear, worry, or unease”. To experience Stress, a stressor must be there, however, Anxiety can happen without any obvious trigger or it can happen as a reaction to Stress. American Psychological Association (APA) informs that Anxiety is defined by “persistent, excessive worries that don’t go away even in the absence of a stressor”. Following these arguments, we ask a question: Is it possible a response that is dependent on external trigger can stimulate a response that is internal? As we have mentioned earlier that Anxiety can happen as a reaction to Stress (Healthline), the same relationship is emphasized by Mental Health First Aid i.e., “Anxiety is a person’s specific reaction to stress; its origin is internal.” Mental Health First Aid further mentions that “Stress is a common trigger for anxiety and it’s important to catch anxiety symptoms early to prevent development of an anxiety disorder.”. According to Medical News Today, “Sometimes, stress can develop into anxiety. Stress is the body’s reaction to a threat, and anxiety is the body’s reaction to stress.”

Based on above discussion, one can conclude that “Anxiety is body’s reaction to Stress which simply means that Anxiety is caused by Stress, but not vice versa”. However, Khozaei et al. (2024) hypothesized an exactly reverse relationship. In their proposed framework, Anxiety causes Stress but in reality, Stress causes Anxiety. This puts the whole “Research conceptual model” in hot water.

Besides the erroneous framework of this study, these issues could have been addressed if the Authors had conducted a thorough Literature Review and underpinned the framework using migratory theories e.g., social isolation, cultural shock, goal-striving stress, cultural change.

It seems that the Authors administered 497 questionnaires and collected data from all respondents (p. 5; see image below). This gives a 100% response rate. We believe that more than 497 surveys were distributed and 497 were received back. However, nothing was reported on how many total surveys were disseminated and how non-response bias was dealt? This study is a single informant study but we can’t find any information on how the Authors dealt with common method variance (CMV) and hence common method bias (CMB)?

On page 5, the Results section mentions “participants were proficient in the Farsi language”. This article doesn’t inform us whether the questionnaires were designed in English or Persian? Since the respondents were proficient in Persian, the language of the questionnaires should be Persian. If it was Persian, no information was given what language translation protocol was followed and how the translated and back-translated versions were processed to avoid problems with the translation validity of the questionnaire? This is indeed a significant question the Authors completely missed to address and Editors and Reviewers failed to raise.

Now we want to draw your focus to a very important issue pertaining to measurement most of the authors falsely deal with. The Authors reported “To measure social cohesion three items from (Scholten et al., 2017, 2018).” Let’s go to the reported citation. We see that in the both cited articles, Scholten and his colleagues tapped perceived wealth, justice, and freedom using single-item questions; they didn’t tap social cohesion or used social cohesion construct in their study. However, Khozaei et al. (2024) tapped belongingness, freedom, and fairness. Therefore, it is wrong to mention that the Authors used the scale from Scholten et al. (2017, 2018).

First of all, we need to understand what social cohesion is. Jenson (2010) included inclusion, equity, legitimacy, participation, recognition, and belonging in their social cohesion measurement framework. Manca (2014) cites that “social cohesion refers to the extent of connectedness and solidarity among groups in society. It identifies two main dimensions: the sense of belonging of a community and the relationships among members within the community itself.” Similarly, Kim, Sheely, and Schmidt (2020) found that political participation, socio-cultural participation, and substantial relations are significant factors of social cohesion. It can be observed that there is a consensus among researchers regarding the “belongingness” dimension of social cohesion construct.

Strangely, Khozaei et al. (2024) ruled out the “belongingness” dimension (due to low factor loading) and only considered freedom, and fairness as the dimensions tapping Social Cohesion which is quite strange. It simply means that after excluding the backbone of Social Cohesion, authors still believe that they tapped the construct. We wonder why was not this question raised by the Editor and Reviewers of the journal?

Let’s see another important issue found in many papers reporting PLS-PM results.

Below, we are reporting the problematic results reported by the Authors (p. 6, 7) along with our opinion on these results in parentheses:

SmartPLS Direct Effects Estimation:

path coefficient = 0.036 ; t-statistic = 3.234, p-value < 0.001 (In our opinion such a low impact should lead to an insignificant t-statistics with values < 1.96 and higher p-values)

path coefficient = 0.900 ; t-statistic = 2.834, p-value < 0.042 (In our opinion such a high impact should lead to a greater value of t-statistics with small p-values)

path coefficient = -0.071 ; t-statistic = 1.754 , p-value < 0.037 (In our opinion such a low impact should lead to an insignificant t-statistics as 1.735 < 1.96 and higher p-values)

path coefficient = 0.089 ; t-statistic = 3.722 , p-value < 0.001 ( In our opinion such a low impact should lead to an insignificant t-statistic with lower values; even if the t-statistic reaches 1.96 or bit higher the p-value should be greater)

path coefficient = -0.088 ; t-statistic = 1.634 , p-value < 0.067 ( In our opinion such a low impact should lead to an insignificant t-statistics as 1.634 < 1.96 and higher p-values)

SmartPLS Indirect Effects Estimation:

path coefficient = 0.098 ; t-statistic = 6.105 , p-value < 0.002 ( In our opinion such a low impact could be significant in the case of indirect effects but t-statistic should be lower with the higher p-value say p < 0.01 but > 0.002)

path coefficient = -0.059 ; t-statistic = 3.586 , p-value < 0.001 ( In our opinion such a low impact should lead to an insignificant t-statistic with lower values; even if the t-statistic reaches 1.96 or bit higher the p-value should be greater)

path coefficient = -0.148 ; t-statistic = 1.586 , p-value < 0.001 ( In our opinion this impact can be significant with p-values greater than 0.01 but t-statistics is 1.586 < 1.96 which is insignificant)

Our source contacted the SmartPLS team to seek their opinion on the above-mentioned results along with our opinion on these results. Below we are appending their response:

“Generally, you are right with your expectation that smaller coefficients should have smaller t-values and larger coefficients should have larger t-values. If the variability of the coefficient is high even a large coefficient can have a small t-value. This often happens when you have strong multicollinearity. Multicollinearity increases the variance of the estimate (that is also why we look at VIF=variance inflation factor). In contrast, smaller coefficients can have small variability (if the relationship is weak but stable) and thus large t-values. For example, the nature of indirect effect usually makes it that they have quite small effects (because they are a product of two coefficients smaller than 1), but their variability is usually also small, so that even small indirect effects can be significant.”

“You are right with your observation that some p-values seem strange given the t-values, especially the last one, where the p-value should be about 0.056 (one-tailed test) or about 0.11 (two-tailed test) given a t-value of 1.586.”

“The path coefficient of 0.900 is quite extreme; indicating either high multicollinearity or a near perfect dependence. In such a case, discriminant validity problems are also often likely.”

“We think it is not a problem of PLS based research, but a problem of research in general. We also see this with experimental research and other types of models and data. Researchers who are desperately in the need for publications do a lot of misconduct. From p-hacking and data manipulation to bluntly faking data and results.”

Based on our opinion and comments from SmartPLS, we can conclude that the reported results put the study and the publishing journal in a predicament. Ohh! we forgot to mention single value confidence intervals (p. 7) reported by the Authors. What could we say now?

In principle, we believe this article should be retracted or an erratum notice should be issued by the Publisher/Editors of the Journal of Migration and Health.

To read the original article, please visit Journal of Migration and Health website by clicking here

References

Jenson, J. (2010). Defining and measuring social cohesion. UNRISD & Commonwealth Secretariat: London, UK.

Manca, A.R. (2014). Social Cohesion. In: Michalos, A.C. (eds) Encyclopedia of Quality of Life and Well-Being Research. Springer, Dordrecht.

Kim, J., Sheely, R., & Schmidt, C. (2020). Social capital and social cohesion measurement toolkit for community-driven development operations. Mercy Corps and the World Bank Group, Washington, DC.

Update: On 27 January, 2024, Dr. Fatemeh Khozaei (first author) sent us a threatening email in response to our article. Please find the contents of the email here: "Your work is very unethical and not based on great understanding and knowledge. You have to remove our paper from your website or I will complain against your unethical website."

Update: On 27 January, 2024, in their second email, Dr. Fatemeh Khozaei (first author) tried to justify Anxiety-Stress relationship by citing a questionable study Spinoni et al. (2023). The Author also struggled to justify few other highlighted mistakes but completely faild to do so. However, the Author doesn't mention anything on other raised issues including flawed data reporting and scale adoption issues.

Click to Read Previous Article