Standardised effect size measurements

If you’re in a field that uses Analysis of Variance, you have surely heard that p-values alone don’t indicate the size of an effect. You also need to give some sort of effect size measure.

Why? Because with a big enough sample size, any difference in means, no matter how small, can be statistically significant. P-values are designed to tell you if your result is a fluke, not if it’s big.

Truly the simplest and most straightforward effect size measure is the difference between two means. And you’re probably already reporting that. But the limitation of this measure as an effect size is not inaccuracy. It’s just hard to evaluate.

If you’re familiar with an area of research and the variables used in that area, you should know if a 3-point difference is big or small, although your readers may not. And if you’re evaluating a new type of variable, it can be hard to tell.

Standardized effect sizes are designed for easier evaluation. They remove the units of measurement, so you don’t have to be familiar with the scaling of the variables.

Cohen’s d is a good example of a standardized effect size measurement. It’s equivalent in many ways to a standardized regression coefficient (labeled beta in some software). Both are standardized measures-they divide the size of the effect by the relevant standard deviations. So instead of being in terms of the original units of X and Y, both Cohen’s d and standardized regression coefficients are in terms of standard deviations.

There are some nice properties of standardized effect size measures. The foremost is you can compare them across variables. And in many situations, seeing differences in terms of number of standard deviations is very helpful.

But they’re most useful if you can also recognize their limitations. Unlike correlation coefficients, both Cohen’s d and beta can be greater than one. So while you can compare them to each other, you can’t just look at one and tell right away what is big or small. You’re just looking at the effect of the independent variable in terms of standard deviations.

This is especially important to note for Cohen’s d, because in his original book, he specified certain d values as indicating small, medium, and large effects in behavioral research. While the statistic itself is a good one, you should take these size recommendations with a grain of salt (or maybe a very large bowl of salt). What is a large or small effect is highly dependent on your specific field of study, and even a small effect can be theoretically meaningful.

Another set of effect size measures for categorical independent variables have a more intuitive interpretation, and are easier to evaluate. They include Eta Squared, Partial Eta Squared, and Omega Squared. Like the R Squared statistic, they all have the intuitive interpretation of the proportion of the variance accounted for.

Eta Squared is calculated the same way as R Squared, and has the most equivalent interpretation: out of the total variation in Y, the proportion that can be attributed to a specific X.

Eta Squared, however, is used specifically in ANOVA models. Each categorical effect in the model has its own Eta Squared, so you get a specific, intuitive measure of the effect of that variable.

Eta Squared has two drawbacks, however. One is that as you add more variables to the model, the proportion explained by any one variable will automatically decrease. This makes it hard to compare the effect of a single variable in different studies.

Partial Eta Squared solves this problem, but has a less intuitive interpretation. There, the denominator is not the total variation in Y, but the unexplained variation in Y plus the variation explained just by that X. So any variation explained by other Xs is removed from the denominator. This allows a researcher to compare the effect of the same variable in two different studies, which contain different covariates or other factors.

In a one-way ANOVA, Eta Squared and Partial Eta Squared will be equal, but this isn’t true in models with more than one independent variable.

The drawback for Eta Squared is that it is a biased measure of population variance explained (although it is accurate for the sample). It always overestimates it.

This bias gets very small as sample size increases, but for small samples an unbiased effect size measure is Omega Squared. Omega Squared has the same basic interpretation, but uses unbiased measures of the variance components. Because it is an unbiased estimate of population variances, Omega Squared is always smaller than Eta Squared.

Odds Ratio Confidence Interval

Following formula is used to calculate the odds ratio (O.R.) and its confidence interval (C.I.). 

OR = a*d / b*c, where:

• a is the number of times both A and B are present,
• b is the number of times A is present, but B is absent,
• c is the number of times A is absent, but B is present, and
• d is the number of times both A and B are negative.

To calculate the confidence interval, we use the log odds ratio, log(OR) = log(a*d/b*c), and calculate its standard error:

se(log(OR)) = √1/a + 1/b + 1/c +1/d

The confidence interval, C.I., is calculated as:

CI = exp(log(OR) ± Zα/2­*√1/a + 1/b + 1/c + 1/d),

where Zα/2 is the critical value of the Normal distribution at α/2 (e.g. for a confidence level of 95%, α is 0.05 and the critical value is 1.96).

Note: The logarithms included in the formulae above are natural logarithms, i.e., log base e, sometimes denoted ln().

Different types of graphs

Line Graphs

Line charts, or line graphs, are powerful visual tools that illustrate trends in data over a period of time or a particular correlation. For example, one axis of the graph might represent a variable value, while the other axis often displays a timeline.

Each value is plotted on the chart, then the points are connected to display a trend over the compared time span. Multiple trends can be compared by plotting lines of various colors.

For example, the interest of digital marketing over time can be visually shown with ease through the use of a line graph. Simply plot each number of searches along the timeline to view the trend.

Bar Graphs

The simplest and and most straightforward way to compare various categories is the classic bar graph. The universally-recognized graph features a series of bars of varying lengths.

One axis of a bar graph features the categories being compared, while the other axis represents the value of each. The length of each bar is proportionate to the numerical value or percentage that it represents. 

For example, $4 could be represented by a rectangular bar four units long, while $5 would equate to a five-unit long bar. With one quick glance, audiences learn exactly how the various items size up against one another.

Bar graphs work great for visually presenting nearly any type of data, but they hold particular power in the marketing industry. The graphs are ideal for comparing any sort of numeric value, including group sizes, inventories, ratings and survey responses.

Pie Charts

Pie charts are the simplest and most efficient visual tool for comparing parts of a whole. For example, a pie chart can quickly and effectively compare various budget allocations, population segments or market-research question responses.

Marketing content designers frequently rely on pie charts to compare the size of market segments. For example, a simple pie graph can clearly illustrate how the most popular mobile-phone manufacturers compare based on the sizes of their user-bases.

Mosaic or Mekko Charts

Basic line, bar and pie charts are excellent tools for comparing one or two variables in few categories, but what happens when you need to compare multiple variables or multiple categories at the same time?

What if all those variables aren’t numeric even? A mosaic – or Mekko – chart plot might be the better choice.

Perhaps a market analyst, for example, wants to compare more than the size of various mobile-phone markets. What if, instead, he or she needs to compare the size of the user bases, as well as the age groups within each group?

A mosaic chart would allow said marketer to illustrate all the variables in a clear and straightforward manner.

In the above example, one axis of the chart represents the categories being compared – mobile phone manufacturers – while the other axis lists various age ranges.

The size and color of each cross-section of the chart corresponds with the market segment it represents, as depicted in the chart's legend.

Population Pyramids

Market segments are often divided based on age and gender, and a population pyramid is an ideal visual representation of the two groups.

The graph classically takes on the shape of a pyramid when a population is healthy and growing -- the largest groups are the youngest, and each gender dwindles somewhat equally as the population ages, leaving the smallest groups at the top of the graph.

A population pyramid that veers away from its classic shape might indicate an irregularity in a population during a particular period, such as a famine or an economic boom that led to an increase in deaths or births.

Of course, population pyramids aren’t always used to compare populations by age, and therefore don’t always take on the graph’s namesake shape.

A marketer, for example, might use the design to compare a population by income, weight or IQ, in which the smallest groups will often be at both the top and bottom. Regardless, the graph clearly depicts population trends, while it compares the sizes of two related groups.

Spider Charts

When a statistician needs to visually compare three or more quantitative variables, he or she might choose to use a  radar chart, also known as a spider or star chart.

The chart usually consists of a series of radii, each representing a different category, that splay out from a center point like spokes.

The length of each “spoke” is proportionate to the value being compared. For each category, the spokes are then connected with a line of a designated pattern or color, forming a star-like shape with points equal to the number of categories.

The result is a graphic representation that can reveal trends and compare categories all at the same time.

 Stock Charts

Image source

One of the most vital of all financial graphs, stock charts help investors track the markets to determine profits and loss, as well as make buying and selling decisions.

While a variety of graphs are used to represent market changes, the most common is likely the basic line graph turned histogram.

The lines simply tracks changes in a particular stock’s or overall market’s value over a period of time. Multiple stocks can be tracked and compared at the same time by transforming the line graph into a stacked area chart or simply using multiple lines of various colors.

Flow Charts

Oftentimes in business – as well as other industries – a process must be diagrammed. A flow chart allows a process to be sequenced step-by-step, from beginning to end, for the purpose of analyzing, designing, documenting or managing it.

These flow charts can even feature multiple beginnings and ends, with countless pathways and journeys in between.

While a simple flow chart can certainly document a basic process from A to B to C, the diagrams are more frequently used to illustrate more complex sequences with multiple decisions or conditions along the way.

Each time a condition is met, the chart diagrams the various options, then the path continues following each choice.

Gantt Charts

Gantt charts are special types of bar graphs used to diagram projects and schedules. The use of colored bars of varying lengths reflect not only a project’s start and end dates, but also important events, tasks, milestones and their timeframes.

Modern Gantt charts can also illustrate activities’ dependency relationships.

If Team 3’s completion of task C, for example, is dependent upon the prior completion of task B by Team 2, the chart can not only reflect that relationship, but the scheduled dates and deadlines for each.

Control Charts

Also commonly known as a process-behavior chart, a control chart helps determine if a data set falls within a mean or predetermined control range.

Frequently used in quality control processes, a typical control chart consists of points plotted on two axes, representing sample measurements.

The mean of each point is calculated, and a center line across the graph at the mean value. Then, a standard deviation from the mean is calculated using each sample.

Finally, upper and lower control limits are determined and diagrammed to reflect the points at which deviation is beyond the expected standard.

Waterfall Charts

Particularly useful in accounting and qualitative analysis, waterfall charts illustrate how an initial value is affected positively and negatively by various factors.

For example, a waterfall chart could clearly and efficiently communicate how an opening balance changes month by month over the course of a year.

Because they often appear as though bars are floating throughout the graph, waterfall charts are sometimes referred to as floating bricks or Mario charts.  

Hierarchy Diagrams

Similar in appearance to a flow chart, a hierarchical diagram, also known as an organizational chart or an organigram, illustrates the structure of an organization, as well as the relationships within it.

A typical company organigram, for example, lists the CEO at the top, followed by presidents, vice presidents, managers and so on.

An organizational chart can illustrate the chain of command from any employee all the way to the top. Hierarchy diagrams are similarly used to represent pedigrees, scientific classifications, demographics and any data set with a similar breakdown.

Take the above diagram as an example, where a project team is organized in an organizational hierarchy chart so that everyone knows who their supervisor is in a project.

Also known as a scattergram, the graph consists of two axes, each representing a set of data. For example, one axis might represent the numbers of miles driven by a vehicle, while the second axis displays the total gallons of gas used.

For each vehicle sampled, its miles-per-gallon average is represented by a dot plotted onto the graph. Once multiple dots are plotted, trends can be spotted and samples can be compared, depending on how many colors are featured in the chart.

Trellis Plots

Sometimes a statistician will need to compare more data sets than can be represented by a single graph. What if, for example, a graph needs to compare not only miles driven and gallons used, but also the number of gears and cylinders contained in each vehicle sample?

A trellis plot, also called a lattice graph or plot, can display and compare all of those variables. While the above example uses a series of scatter charts, trellis plots commonly feature series of bar or line graphs, as well.

Function Plots

Mathematicians, engineers and statisticians often need to determine the value of an equation by graphing its result. The graph of a function is the set of all points whose coordinates satisfy the equation.  

Therefore, the function of an equation with variables of x and y would be drawn on a graph with an x and y axis. Likewise, an equation that also included a variable of z would need to be drawn on a three-dimensional graph with a third axis.

Function graphs of common shapes are visually associated with their corresponding algebraic formulas.

Binary Decision Diagrams

A binary decision is a choice between two alternatives, so a binary-decision diagram illustrates the path from one decision to another.

In computer science, binary decisions make up the Boolean data type, in which two values are associated with different actions within a process flow.

Outside of computer science, a binary-decision diagram can still be used to illustrate any process by which actions are based on a decision between two values, whether those conditions be yes or no, true or false, 1 or 0 or any other opposing choices.

Ultimately, the path taken will diagram how the process flowed, from beginning to end

 

Timelines

Possibly the most self-explanatory of data visualizations, a timeline tracks data over a time period. Significant dates and events are highlighted at the point at which they appear on a chronological scale. Timelines can be used alone or in conjunction with other visualizations.

A form of hierarchical diagram, a genealogical tree illustrates the structure of a family. It can either begin with an ancestor, then diagram his or her descendants, their siblings, marriages and children, and so on.

A pedigree chart, on the other hand, begins with an individual and charts their ancestry, from parents to grandparents, and continues up.

Sunburst Charts

A type of multi-level pie chart, a sunburst chart is used to illustrate hierarchical data using concentric circles. Each ring of the “sunburst” represents a level in the hierarchy, with the root node represented by the center circle, and the hierarchy moving outward.

While a sunburst chart can be used to illustrate a familiar or company hierarchy, it can also break data down by time periods, creating a historical hierarchy.

Various branches of an organization can be represented by designated hues, with different levels often taking on varying shades of the same color family. Rings can also be divided further to represent multiple divisions within the same organizational level.

In fact, a traditional, complex color wheel, such as that used by paint stores, is another form of sunburst chart.

Line Graphs

If a timeline is a form of graph, then it only makes sense that historians often employ it in displaying other data. By plotting immigration levels against a timeline, the resulting histogram illustrates population trends over a century or longer with a basic line graph.

Stacked Area Charts

Stacked area chart are frequently used to diagram changes of multiple variables across time. Multiple lines can be drawn, for example, to track the population changes of various states across time.

The area below each line can be colored a different hue to represent the state it signifies, resulting in a graph that clearly represents population trends, while at the same time displaying each state’s data in order from least to most populous.

 

Stacked Bar Graphs

When studying groups of people, it’s common to compare multiple variables at once. After all, it’s enormously more useful to examine racial backgrounds, ages and gender in addition to total population.

A stacked bar graph combines elements of the traditional bar graph and the pie graph to communicate totals, trends and proportions in a single illustration.

Rather than simply illustrating changes in global population over time with a traditional column bar graph, a stacked bar graph can also represent the racial makeup of the population during each year and how those proportions have changed during the same period.

Trellis Bar Graphs

When presenting data with three variables, a designer might try and create a three-dimensional bar graph, but adding an additional axis can sometimes appear cluttered and unclear, especially in printed form.

Instead, additional variables can be presented in a trellis – or lattice – format.

By combining a series of bar graphs in a modular design, additional sets of data can be easily compared. For example, a single bar graph could illustrate the political breakdown of Poland’s national elections over a period of five years.

But a trellis bar graph could depict the same data set for 16 European nations.

Stacked Area Charts

Stacked area charts are ideal for comparing values that would normally require multiple line graphs. Each line represents a different category, and the area below each line is generally shaded a designated color so each data set can be easily compared.

For example, an area chart with one axis that represents a numeric value, and another axis that serves as a timeline, data for various categories over time can be tracked and compared with a single graphic. 

Multi-level Pie Charts

All too often a designer finds him or herself with more sets of data than can be presented in a single standard graph. Fortunately, in the case of a pie chart, multiple layers of data can be presented without the need for multiple images or a trellis design.

A multi-level pie chart, for example, consists of tiers, with each layer representing a separate set of data, and can be the perfect solution.

So while it would take three traditional pie graphs to illustrate the various sources of recorded words for three different decades, a multi-level pie graph can not only take the place of all three, but it also offers a clearer visual comparison of each year’s results.

Venn Diagrams

The classic Venn diagram, also known as a logic diagram, illustrates all possible logical relationships between a designated collection of sets.

For example, the overlap of two or more circles – in this case there are three – visually represents the similarities and differences between the social, economical and environmental areas of sustainable development.

The more circles used, the more logical conclusions that can be represented by their overlap. The combined set of all data in the diagram is known as the union, while the areas that overlap are called intersections.

A Venn diagram in which the relative size and area of each shape is proportional to the size of the group it represents is known as an area-proportional or scaled Venn diagram.

 

Science

Scattergrams

Scattergrams, also known as scatter plots, are graphs that show the relationship between two or more variables. The plots use mathematical coordinates to represent two variables of a data set.

Data is displayed in a scattergram as a collection of points, each representing the value variables plotted on a horizontal and vertical axes. If points are color-coded, an additional variable can be represented in a single chart.

By plotting certain data sets, scientists can discover trends of which they might not otherwise be aware. For example, a scattergram might allow a doctor to plot patients’ resting heart rates against their body-mass index figures.

The resulting graph reveals that a higher heart rate correlates with a higher BMI.

Trellis Line Graphs

Trellis graphs allow scientists to examine complex, multi-variable data sets, comparing a greater deal of information at once.

While a single line graph can illustrate monthly UFO sightings in Tennessee over an 18-year period, a trellis line graph will display the same data for all 50 states in a single graphic.

A trellis line graph is based on the same principle as its simpler counterpart, plotting trends in a dataset consisting of two variables – numbers of UFO sightings and dates – through use of connecting points on two axes.

But by combining multiple line graphs in a modular format, an additional variable – location – is represented.

Pareto Charts

Sometimes a basic graph doesn’t display enough information to draw the necessary conclusion. A Pareto chart combines a bar graph with a line graph to illustrate not only categories’ individual values, but also the cumulative total of the entire set.

Pareto charts are designed to highlight the most important of a set of factors.

In a Pareto chart that tracks the type and frequency of food defects, the bars illustrate each type of defects’ total occurrences – as reported on one of the charts’ axes – while the line charts the cumulative frequency of all categories, from most to least prevalent.

The result is a graph that clearly reflects the most common food defects and what percentage of the whole each represents.

Radar Charts

A radar chart, also commonly referred to as a spider chart or a star chart, displays data sets consisting of three or more variables on a two-dimensional graphic. Each variable’s quantitative value is reflected across an axis that usually starts in the chart’s center point.

As each item’s variables are charted, a line connect the points on each axis, forming an irregular polygon that may or may not resemble a star or spider web.

Multiple data sets can be compared on a single radar graph by representing each with a different color, identified by labels or in an accompanying key.

A radar chart can, for example, clearly compare and illustrate the costs and outcomes of various medical procedures as they relate to multiple conditions – all in a single graphic.

Spherical Contour Graphs

Image source

Plotting planetary conditions on a basic two-axis graph can pose a problem. The Earth, after all, is a sphere. Instead, data can be plotted on a three-axis field using variables of x, y and z. The resulting plot, if completed, will take the form of a sphere.

A spherical plot can, for example, reveal global temperature or rainfall trends by assigning each value range with a particular color, then plotting the data with points of the corresponding hue.

 

Health and Wellness

Multi-Line Graphs

Just as medical symptoms are rarely isolated, neither is the analysis of biometric data. After all, rarely does one statistic paint the entire medical picture.

Line graphs can reflect multiple data sets with lines of varying patterns or color. For example, a multi-line graph can illustrate changes in life expectancies of not just the population in general, but for each gender and multiple racial backgrounds.

Stacked Bar Graphs

Stacked bar graphs aren’t useful only in illustrating parts of of a whole. They can also be used to display additional variables.

While a basic bar graph could represent what portion of a population is classified as overweight over a designated time period, a stacked bar graph can also track how much of the total is obese.

Flow Charts

Following the proper process is probably more important in medicine than in any other field. After all, if the surgeon forgets a step, you might very well bleed to death while you sleep.

Flow charts are frequently used by hospitals, clinics and other medical facilities to ensure proper procedures are uniformly followed.

Pictograms

In a pictogram, or pictograph, images and symbols are used to illustrate data. For example, a basic pictogram might use an image of the sun to signify each fair-weather day in a month and a rain cloud to symbolize each stormy day.

Because images are known to hold more emotional power than raw data, pictograms are often used to present medical data.

An illustration that shades five of 20 person symbols to represent a 20-percent death rate carries a more powerful message, for example, than a bar, line or pie that illustrates the same data.

Anatomical Diagrams

Medical diagrams are often used to illustrate anatomy, treatments or disease pathology in order to explain treatments for patients and others without an extensive biomedical background.

While medical diagrams are considered a combination of science and art, they can be just as technical as any other quantitative graph. And no matter how detailed the drawing, anatomical diagrams are designed to clearly and efficiently present data.

And just as with a complex contour diagram, the diagrams focus on key information, even if it was selected from voluminous amounts of medical or scientific data.

Multi-Pie Charts

Just as in the cases of multi-level pie graphs, stacked bar graphs and trellis plots, multi-pie graphs paint a more detailed portrait of the data set it illustrates.

While a single pie chart can display what portion of the total population has a particular condition, a multi-pie graph can break those statistics down to illustrate not only the portion of men and the portion of women, but also how the two groups compare.

Image source

It can be difficult to graphically represent medical data sets that consist of hundreds -- or more -- patients, as is the case in most medical studies.

But a scatter plot allows for the representation of each subject, plotted on the graph according to the variables on the chart’s two axes.

The pattern formed by the plotted dots can clearly determine trends in the data. By analyzing a scatter plot, for example, a researcher could easily identify a correlation between longer life expectancy and higher household income.

 

Meteorology and Environment

Contour Plots

Image source

Contour plots allow for the analysis of three variables in a two-dimensional format. Instead of plotting data along two main axes, the graph also presents a third value that is based on shading or color.

Just as a topographical map plots longitude, latitude and elevation in a two-dimensional design, a contour graph illustrates values of x, y and z.

With a contour graph, for example, a climatologist can not only plot ocean’s salinity on different dates, but its salinity at various depths on those dates.

Heat Maps

Image source

A type of contour graph, a heat map specifically charts varying temperatures at different geographical points. While the graph’s two axes are a map’s latitude and longitude, the third variable – temperature – is represented by a spectrum of color.

While most commonly used to illustrate weather, heat maps also can represent web traffic, financial indicators and almost any other three-dimensional data.

Scatter-Line Combo

By combining a line graph with a scatter plot, meteorologists and other statisticians can illustrate the relationship between two data sets.

For example, the high and low temperatures of each day in a month can be displayed in a scatter plot, then a line graph can be added to plot the historic average high and low temperatures over the same period.

The resulting combination graph clearly displays how the temperature range each day compares to the historic average, and it even indicates how those measurements trend over the examined time period.