[SNA & Statistics] Why Standard Statistics Fail on Network Data
In social network analysis, you inevitably face questions like:
“Is this result really meaningful?” “Does the structure of the network have a statistically significant effect on the outcome?” To answer these questions, statistical inference is necessary. Naturally, you might first think of familiar methods such as t-tests or regression analysis. However, there is a critical problem. Network data is built on relationships, not independent observations. Because of this, it often violates the basic assumptions of standard statistical methods. So how should we test significance while properly accounting for network relationships? In this post, we explain why applying ordinary statistics directly to social network analysis can be problematic, and introduce alternative statistical approaches that take relationships into account.
Why does network analysis need a different statistical approach?
Network data is not independent.
Most standard statistical methods assume that observations are independent and identically distributed.
In a survey, for example, one person’s response is assumed not to affect another’s response.
Only when this assumption holds do sample size, standard errors, and p values have clear meaning.
The problem is that network data does not satisfy this assumption.
In social network analysis, the basic unit of data is not individuals, but relationships.
And these relationships are tightly interconnected.
2. Network data is structurally interdependent
In network data, a connection between two actors affects the likelihood of other connections. For example, if A frequently communicates with B, B is also more likely to communicate with A. If A is connected to C, then B and C are also more likely to become connected. Because of this, network data shows structural patterns such as reciprocity, transitivity, and clustering. As a result, observations in a network are not independent. They are strongly correlated with each other.
This problem becomes especially serious when we think about sample size.
Imagine analyzing a network of 50 people using an adjacency matrix. The matrix contains 2,500 cells (50 × 50). In standard statistics, these are treated as 2,500 independent observations. But in reality, this data represents a single relational structure created by 50 actors. It is hard to say that all 2,500 cells contain independent information. This is like thinking you analyzed 100 survey responses, only to find out that the respondents actually belonged to 10 families and gave almost identical answers. There is a large gap between the apparent sample size and the real amount of information.
3. Distorted statistical results
What happens if we apply standard regression or correlation analysis in this situation? Because the sample size is perceived as much larger than it really is, standard errors are underestimated, t values become unusually large, and p values are calculated to be far smaller than they should be. As a result, relationships that are actually close to random can appear highly statistically significant. A study has shown that when ordinary OLS regression is applied to network data, more than half of completely random datasets can still produce “significant” results.
(Source: Krackhardt, D. (1988). Predicting with networks: Nonparametric multiple regression analysis of dyadic data. Social networks, 10(4), 359-381.)
How network structure creates bias
To be statistically rigorous, it is not enough to simply say that the IID assumption is violated.
We need to understand how and why the data becomes biased.
The key concept here is network autocorrelation. Network autocorrelation refers to the tendency for an actor’s attributes or outcomes to resemble those of their connected neighbors. If autocorrelation over time means “yesterday’s value affects today’s value,” network autocorrelation means “connected people influence each other’s characteristics.”
This phenomenon appears in two main ways. The first is network disturbances, where factors not explicitly included in the model are shared through the network. People in the same group often share similar environments or information, which makes the error terms themselves correlated. Example: - You model students’ grades (the dependent variable) using study time (the independent variable). - Students in the same study group may share tips or be exposed to the same noise, even without realizing it. - As a result, the regression residuals are not independent but clustered. This does not necessarily bias the regression coefficients, but it does distort the confidence intervals.
(Source: https://home.cc.umanitoba.ca/~godwinrt/7010/topic8.pdf)
The second is network effects, where the network structure itself directly influences the outcome. Example: - An individual’s decision to adopt an innovation is directly affected by whether their friends adopt it. - If standard regression is used, peer influence may be wrongly attributed to other explanatory variables, or the true effect of those variables may be underestimated.
(Source: Neuman, E. J., & Mizruchi, M. S. (2010). Structure and bias in the network autocorrelation model. Social Networks, 32(4), 290-300.)
This issue is closely related to a long-standing problem in anthropology known as “Galton’s problem.” When similar patterns appear, it is hard to tell whether they come from the same underlying cause, or whether they spread simply because people are close to each other. The same challenge appears in network analysis. It is difficult to determine, using standard statistics alone, whether a behavior is caused by network influence or by similar people choosing to connect with each other in the first place. For example, why is the smoking rate high in a friend group? Is it because friends influence each other to smoke (influence effect), or because smokers tend to associate with other smokers (selection effect)? In the next post, we will look at statistical methods designed for network analysis that help distinguish between these possibilities.
[Related Articles]
References
Cyram (2025). NetMiner 5 Manual. https://netminer.gitbook.io/netminer5
Wasserman, S., & Faust, K. (1994). Social Network Analysis: Methods and Applications. Cambridge University Press.
Hanneman, R. A., & Riddle, M. (2005). Introduction to Social Network Methods. University of California, Riverside. (Online Book)
Borgatti, S. P., Everett, M. G., & Johnson, J. C. (2018). Analyzing Social Networks (2nd ed.). SAGE Publications.
Krackhardt, D. (1988). Predicting with networks: Nonparametric multiple regression analysis of dyadic data. Social Networks, 10(4), 359-381.
Dekker, D., Krackhardt, D., & Snijders, T. A. B. (2007). Sensitivity of MRQAP tests to collinearity and autocorrelation conditions. Psychometrika, 72(4), 563-581.
Simpson, W. (2001). QAP: The Quadratic Assignment Procedure. North American Stata Users' Group Meetings 2001.
Leenders, R. T. A. J. (2002). Modeling social influence through network autocorrelation: Constructing the weight matrix. Social Networks, 24(1), 21-47.
Mizruchi, M. S., & Neuman, E. J. (2008). The effect of density on the power of the network autocorrelation model. Social Networks, 30(3), 190-200.