2. DATA SOURCE AND MEASURES

2.1 Measures of literacy and numeracy in ALL

The ALL survey collected data from a representative sample of approximately 7,100 adults who were aged from 16 to 65 years old during the 2005-2007 period.^[1] The target population was usual residents who were living in private households. The New Zealand ALL survey was carried out as part of an international data collection. The New Zealand survey was very similar to the survey conducted in other countries, although some minor changes were made to reflect New Zealand vocabulary, conventions and institutions.^[2]

Four skill domains were measured in ALL:

• Prose literacy is the ability to read and understand continuous text, such as news stories or instruction manuals.
• Document literacy is the ability to read and understand discontinuous text, such as instructions, graphs, maps and tables.
• Numeracy is the ability to understand and process mathematical and numerical information.
• Problem solving is the ability to plan, reason and solve problems in situations where no routine procedure exists.

These skills were measured directly, by giving survey participants questions to answer and problems to solve, based on written material that they had to read. Each respondent answered a booklet of questions designed to measure their skills on one, two or three of the four skill domains. Based on their question responses, literacy, numeracy and problem-solving scores were assigned to each individual using statistical methods and models that were developed internationally for the survey. A measure of each respondent's level of skill on the domains they were not tested on was imputed using a statistical model of the distribution of skills in the population.

People with different levels of literacy or numeracy skill using the ALL measures can be grouped into five broad skill levels, where level 1 is lowest and level 5 is highest. Appendix 1 provides further information on the types of tasks that can be successfully carried out by people at each skill level.

The objective of ALL was to assess English-language skills, and therefore the questionnaire and assessments were conducted in English. People who were not fluent in English may have performed relatively poorly even if they were highly literate in other languages.

This paper analyses literacy and numeracy skills but does not cover problem-solving skills.

When considering literacy skills, the ALL measure of document literacy is used in this paper rather than the prose literacy measure, because the set of skills covered by document literacy is believed to better match the types of reading that are required at work in many jobs (Sutton, 2009, p.15). However, the prose and document literacy measures in ALL are distributed in a similar way, and analyses using prose literacy tend to give similar results to analyses using document literacy.

In this paper, 'low literacy skills' is defined as having document literacy skills that were assessed at level 1 or level 2 on the five-level ALL scale. 'Low numeracy skills' is defined as having numeracy skills at level 1 or level 2 on the five-level ALL scale. The skills of people at level 1 are more limited than those of people at level 2.

2.2 Interpreting the ALL measures of literacy and numeracy

The fact that a worker has literacy or numeracy skills at level 1 or 2 on the five-level ALL scale does not necessarily mean that they are unable to perform their job in a satisfactory manner at present. Some jobs do not require much use of literacy or numeracy skills, and some workers whose literacy skills are low may still have the essential knowledge that is required for the tasks they need to carry out at work. As a general rule, however, people whose foundation skills are low are more likely than people with higher skill levels to have difficulties with tasks at work that require reading, writing or maths, with learning new knowledge and skills or with adapting to changes at work.

Evidence from employer surveys and firm case studies indicates that low literacy and numeracy skills can lead to mistakes, wastage and output quality problems within production processes, to difficulties with documentation and to an increased risk of accidents (Schick, 2005; Benseman and Sutton, 2007). It seems likely that, in workplaces where a significant number of people have low literacy and numeracy skills, labour productivity is reduced (Workplace Productivity Working Group, 2004).

2.3 Study population

This report focuses on people who were employed at the time they were interviewed. Approximately 5,070 respondents were employed when interviewed for ALL, and the analysis in this paper is based on that sub-sample. The self-employed are included in the study sample along with wage and salary earners, and for that reason, we use the term 'worker' rather than 'employee'.

2.4 Industry and occupation definitions

In the ALL survey, the industry of each employed respondent was coded to the International Standard Industrial Classification (ISIC Rev 3.1) at the two-digit level. In this paper, the ISIC codes were combined into 18 industry groups, which correspond roughly but not perfectly to the ANZSIC 2006 industry group at the two-digit level. A full correspondence to ANZSIC 2006 could not be achieved because of classification differences between ISIC and ANZSIC and because the small number of respondents in a number of two-digit industries (particularly in the manufacturing sector) meant that these industries had to be combined or excluded from the analysis. Details of the 18 industry groups can be found in Appendix 2.

The occupation of each employed respondent was coded to the International Standard Classification of Occupations (ISCO) at the four-digit level. In this paper, these occupational codes were reclassified to NZSCO 1999 at the two-digit level. To ensure that all occupations had adequate sample sizes, some related occupations were combined, giving a final total of 18. Details of the occupational groups used in the paper can also be found in Appendix 2.

2.5 Representativeness of the ALL sample

The survey's target population was adults aged 16-65 years who were living in private households. A 64 percent response rate was achieved (Strafford, 2009). Hence, the level of non-response was significant. The sampling weights developed for the survey were designed to compensate for the effects of non-response. These weights were aligned to Census population benchmarks for five-year age groups, genders and ethnic groups in order to make the weighted estimates calculated from the survey sample as accurate as possible.

Table 1a: Demographic profile of the ALL and HLFS samples, weighted

	ALL		HLFS
	Working age population (16-65 years)	Employed (16-65 years)	Working age population (16-65 years)	Employed (16-65 years)
Mean age	39.1	40.7	39.0	39.6

 

Table 1b: Demographic profile of the ALL and HLFS samples, weighted

	ALL		HLFS
	Working age population (16-65 years)	Employed (16-65 years)	Working age population (16-65 years)	Employed (16-65 years)
	Percent
Female	51.3	47.1	50.8	46.3
Youth - aged 16-24	19.4	12.3	19.8	16.3
Older - aged 55-65	16.8	16.1	16.9	15.2
Maori ethnic affiliation	12.0	10.8	10.8	9.6
Pacific ethnic affiliation	5.5	4.5	5.7	4.7
Born in New Zealand	73.1	75.6	74.7	76.8
Recent immigrant to New Zealand	9.8	8.1	9.6	8.5
5th form/year 11 education or less	25.4	21.9	31.3	27.3
Upper secondary school education only	18.4	15.4	16.6	15.3
Tertiary qualification below degree level	33.5	36.7	34.8	37.9
Degree qualification	22.8	26.0	17.2	19.5

Note: The Household Labour Force Survey results shown in this table were obtained by pooling the data for the four quarters from April 2006 to March 2007.

To assess the representativeness of the final weighted sample, the socio-economic characteristics of ALL respondents are compared with those of Household Labour Force Survey (HLFS) respondents in Tables 1 and 2. We use the HLFS sample as a benchmark because the HLFS is a relatively large household survey with a very similar target population, a high response rate (around 85-90 percent) and population-benchmarked weights, and the HLFS was in the field at the same time as ALL. We report the characteristics of HLFS respondents during the four quarters from 1 April 2006 to 31 March 2007 - the period when most of the ALL interviews were carried out.

On most of the socio-economic characteristics considered here, such as gender, age structure, ethnic composition and native/immigrant mix, the ALL sample is quite similar in profile to the HLFS sample after weights are applied. There are some moderately large differences in educational attainment, however. The ALL sample contains a smaller proportion of adults at the lowest qualification level (5th form/year 11 or below) and a higher proportion of adults with degrees. This is true both for the entire working-aged population and for the sub-sample who were employed. For example, 22 percent of employed adults in ALL but 27 percent in the HLFS were in the lowest educational category. At the other end of the educational scale, 26 percent of employed adults in ALL had degrees, compared with 20 percent in the HLFS.

Table 2: Job profile of workers in the ALL and HLFS samples, weighted

	ALL	HLFS
	%	%
Self-employed	18.6	16.0
Occupational group - NZSCO 1999
Legislators and managers	10.9	13.0
Professionals	17.9	16.6
Technicians and associate professionals	14.3	12.3
Clerks	13.8	12.4
Service and sales workers	14.7	14.4
Agricultural and fishery workers	6.8	6.7
Trades workers	8.5	10.3
Plant and machine operators	8.4	8.4
Elementary occupations	4.8	5.6
Not specified	0.1	0.4
Industry group - ANZSIC 2006
Agriculture, forestry, and fishing	7.7	6.8
Mining	0.4	0.3
Manufacturing	13.4	13.1
Electricity, gas, water	0.8	0.4
Construction	7.4	8.9
Wholesale trade	3.0	4.6
Retail trade	10.8	12.6
Accomodation, cafes and restaurants	5.6	4.6
Transport and storage	3.8	3.8
Communication	2.4	1.8
Finance and insurance	3.1	3.4
Property and business services	10.7	11.5
Government administration and defense	4.8	4.2
Education	10.4	7.8
Health and community services	10.2	9.3
Cultural and recreational services	2.9	2.4
Personal and other services	2.7	4.1
Not specified	0.3	0.4

Note: The Household Labour Force Survey results shown in this table were obtained by pooling the data for the four quarters from April 2006 to March 2007.

The fact that fewer ALL respondents than HLFS respondents are in the lowest educational attainment categories is likely to be partly due to differences in the questions asked on secondary education. ALL asked respondents whether they had completed particular levels of secondary schooling or not, while the HLFS records the qualifications that were attained. Someone who completed year 12 but gained no qualifications at that level would be classified to a higher educational level in ALL than in HLFS. However, questionnaire differences are unlikely to explain the higher proportion of tertiary-qualified people in the ALL sample.

Consistent with the higher educational profile of the ALL respondents, they were also more likely than HLFS respondents to be employed in professional or associate professional and technical occupations and less likely to be employed in trades or elementary occupations.

In summary, the final weighted ALL sample appears to be fairly representative of the target population on most criteria, but adults with low educational attainment appear to be somewhat under-represented and adults with high educational attainment appear to be over-represented. This could lead to some minor biases in the ALL results for outcomes that are highly correlated with educational attainment. For example, given the fact that more highly qualified people are more likely than the less qualified people to undertake further education and training, the ALL estimate of the total proportion of adults who participated in further education and training could be somewhat on the high side. This issue is worth noting when ALL results on education and training participation are discussed below.

---