Today, we will analyze an article titled “The more you connect, the less you connect: An examination of the role of phubbing at home and job crafting in the crossover and spillover effects of work–family spousal support on employee creativity” published in Wiley’s Journal of Occupational and Organizational Psychology (Clarivate SSCI Impact Factor for 2022: 6.2; Scopus Q1).

Exact identification of levels in multilevel research is crucial to get accurate results in multilevel studies. The purpose of multilevel analysis is to investigate the impact exerted by higher-level variables on the lower-level study variables. If the identification of levels is flawed, the overall objective of multilevel analysis cannot be achieved.

On page 12, Wang et al. (2024) mentioned under the sub-heading “Analytical Strategy”:

"Given the nested nature of our study (i.e., daily responses were nested within participants), the daily level is the first level and the between participants level is the second level."

Apparently, the statement seems right but once we read the authors mentioned in Abstract (p. 1):

"Daily diary data were collected from 65 dual-earner couples, over 15 working days in the United States."

And on page 12, the Authors mentioned:

"The APIM allows us to test the mutual effects between the members in the dyad…".

It simply means that, in the given hierarchy, the daily assessments were nested within participants, and participants were nested within dyads. In this case, daily assessments reside at level 1, participants at level 2, and dyads at level 3. This is very important to note that the Authors didn’t notice that they disregarded a complete hierarchical level, therefore, the level 3 equation was missed and was not included in the multilevel model. Hoppmann and Klumb (2006) tested dual-earner couples in their diary study and formed a 3-level hierarchical model. They mentioned on Page 889:

"The first level concerns repeated assessments in daily life, which are nested within individuals (second level) who in turn are nested within couples (third level)."

Hence, we conclude that Wang et al. (2024) tested a flawed hierarchical model because it does not align with their presented hierarchy of data structure.

Now, we will discuss the actor-partner interdependence model (APIM) the Authors used to justify the "crossover within the home domain" segment of their study framework (Figure 1, p. 5). We know that the essential form of APIM (also called stability-influence model) proposed by Cook and Kenny (2005) is a longitudinal/time lagged model in which data for actor's IV – DV (same variable) should be captured at two times. Similarly, data for partner's IV - DV (same variable) should be tapped and their direct and interactional effects are estimated. Other models (nonessential forms) proposed within APIM framework e.g., accuracy-bias model, relational model, and personality model propose that the impact of same actor-partner IVs, could be tested on different actor-partner DVs. However, Wang et al. (2024) placed two same variables as IVs and two same variables as DVs but each predictive pair was tapped by opposite gender. We are not saying this could not happen within the APIM framework but we have two observations: 1. The proposed model does not coincide with the essential form of APIM as proposed by Cook and Kenny (2005) or Kenny et al. (2006) because the model is not longitudinal. 2. The model is not in accord with the other three nonessential forms of APIM. We observed that Authors did not mention anywhere why they proposed this model which is not in accord with above mentioned essential and nonessential forms of APIM?

Let’s look at the scales' reliability estimation method adopted by Wang et al. (2024). The Authors calculated average alpha value across days for all scales they administered on a daily basis. Nezlek (2012) informed that for diary studies, it is wrong to calculate within person means for scale item and calculating alpha value using these scores at person level or calculating alpha for each day and then combining all days to get a single average value. Nezlek and Gable (2001) advised to construct item level (L1), day level (L2), and person level (L3) equations and then estimate item-level reliability of scale by solving the item-level equation intercept. This will indicate how consistent the responses are within days and persons. The estimated value of item-level intercept will be considered as equivalent to alpha reliability in diary type multilevel studies.

We wonder why Editors and Reviewers fail to identify these major mistakes. We hope our readers will not commit such mistakes while conducting multilevel studies.

Don't hesitate to comment if you find this article interesting.

To read full article, please click here.

References

Cook, W. L., & Kenny, D. A. (2005). The actor–partner interdependence model: A model of bidirectional effects in developmental studies. International Journal of Behavioral Development, 29(2), 101–109.

Hoppmann, C. A., & Klumb, P. L. (2006). Daily goal pursuits predict cortisol secretion and mood states in employed parents with preschool children. Psychosomatic Medicine, 68(6), 887–894.

Kenny, D. A., Kashy, D. A., & Cook, W. L. (2006). Dyadic data analysis. In D. A. Kenny (Ed.), (Series Ed.) Methodology in the social sciences. Guilford Press.

Nezlek, J. B. (2012). Multilevel modeling analyses of diary-style data. In M. R. Mehl & T. S. Conner (Eds.), Handbook of research methods for studying daily life (pp. 357–383). The Guilford Press.

Nezlek, J. B., & Gable, S. L. (2001). Depression as a Moderator of Relationships between Positive Daily Events and Day-to-Day Psychological Adjustment. Personality and Social Psychology Bulletin, 27(12), 1692–1704.

Update: On 16 April, 2024, Professor Julie Gore (Editor, Journal of Occupational & Organizational Psychology) contacted us and assured to initiate investigation and corrective action. We look forward to see update and corrective action.

Click to Read Previous Article