Benchmarking in social housing
Benchmarking is a way of evaluating an organisation’s performance against an external standard, or with other organisations. Benchmarking can also be carried out internally by comparing performance measures against a set of standard metrics over time or comparing different departments.
From a social housing perspective, benchmarking is typically used to compare performance against similar organisations, with the aim to share the best practice from the higher-performing organisation to the lower-performing organisation. Learning in this way ensures tried and tested practices are implemented, proving an efficient way to avoid pitfalls of performance improvement initiatives that ultimately fail.
Social housing benchmarking is done in a variety of ways, the most popular and widespread is through the national STAR Survey compiled and reported independently by HouseMark. Organisations also carry out smaller benchmarking exercises with local housing providers, sometimes known as “nearest neighbours”, or through best practice groups set up by housing discipline (eg Assets, Income).
Organisations can also benchmark internally by comparing one department’s performance again another’s – for example, in areas of customer satisfaction, right first time, and operating costs.
Why is benchmarking in the social housing sector unique?
Social housing is a small sector with conflicting pressures in terms of reduced public funding and rising customer expectations. The social housing sector is also separate to, but contained within, the broader supply and demand of the UK housing market. This set of unusual market circumstances means that social housing providers have a limited number of peers nationally with which to conduct a meaningful performance comparison.
Finding comparator organisations within social housing that are a good match for stock size, structure, tenure mix and location can be nearly impossible. When an organisation is a good match, and is getting good performance results, it provides an ideal learning opportunity to share best practice. It is for this reason that benchmarking within the sector is valued by social housing providers.
Benchmarking within vs outside the sector
External benchmarking can be done either with other housing providers, or with other businesses or organisations, as a point of reference. While benchmarking to other organisations outside of housing can prove interesting, benchmarking within the sector avoids false comparisons to purely commercial sectors such as banking or the automotive industry, and to government agencies such as in the health service or welfare systems. Social housing organisations balance a unique set of relationships as landlords, asset managers, and neighborhood partners to local councils. This mix of responsibilities and pressures means that sector benchmarking is the best way to compare and understand performance, by comparing apples with apples (as closely as possible).
The benefits of benchmarking
Benchmarking is an efficient way to improve services by learning from top performing providers, and can help your organisation to understand what good “looks like”. It creates a structure where you measure your performance levels, then compare them with other social housing organisations with similar characteristics to your own. Once these comparisons are complete, you can see where you are doing well, where you are meeting the sector norms, and where you are falling short. To improve poor-performing services quickly, you can learn directly from the high-performing organisations about what they are doing well and how they are getting their good results.
The pitfalls of benchmarking
Although there are a lot of benefits to benchmarking, there are some thinks to note before embarking on your benchmarking journey.
1. Time and resources are critical
Benchmarking takes a long time to complete (sometimes a year), and the process is iterative, typically taking place annually.
The process starts when you create your research programme, aligning to the standards set out in your benchmarking group.
From there, you gather the data from your residents according to representative sample requirements, using survey questions and responses as agreed.
After submitting your benchmarking response scores to your benchmarking group, scores are analysed and the top performers in each group are identified.
These peak performers then share best practice within the benchmarking group. This can be in the form of briefing papers, webinars, or events.
It is up to each benchmarking organisation to decide to what extent they will share their own policies and practices, and adopt best practice from others. But, just because one organisation has success doesn’t mean another will; that is sometimes down to resources, culture, strategy, or other variables that aren’t part of the benchmarking exercise.
2. Reliability and trust are crucial
Benchmarking data isn’t always accurate. Unless you’re comparing apples with apples, the reliability of benchmarking reports can be called into question. The extent that your data gathering, score analysis and submission is exactly the same as the others in your benchmarking group can be up for debate and could cast doubt on the entire process.
Firstly, the method by which your performance data is collected is important. In-house research teams that collect performance data, such as customer satisfaction can have inflated scores due to bias. Secondly, online or SMS data collection can skew towards younger residents who are comfortable with technology, where a telephone-only survey could skew toward older or adults not in employment, education or training. Collection of data by an independent, impartial research agency can provide assurance to your benchmarking group that your data does not contain bias toward any particular result or resident subgroup.
3. The results might not be as positive as you’d like
Finally, after a year of collecting data, analyzing results and submitting your scores to your group, your performance might be very poor. Questions will the asked. The last thing you need are doubts around how the benchmarked information was gathered and analysed. An independent third-party research agency can put any hesitation around the validity of your results to bed, and allow you to get to the important work of improving services for the residents that depend on you.
Best practice for benchmarking in social housing
Benchmarking is more than submitting your performance data to a benchmarking group for comparison. It starts by making sure you (and the members of your group) are all measuring the same things, in the same way using these best practice principles.
The importance of question wording in benchmarking
Question preparation is important when surveying residents, to avoid questions that are leading or contain bias. Questions need to be agreed in advance by the benchmarking group or communicated by a central body like HouseMark. Every effort should be made to ask the survey question as it is worded without deviation from the agreed version.
The importance of response codes in benchmarking
Just as the question wording must be agreed, the response scale and response codes must be identical amongst the benchmarking group participants. The extent to which a resident is “satisfied” or “happy” can record completely different responses, and the inclusion or exclusion of neither, not applicable, and unable to answer accounts for all types of responses and can save the survey from being incomplete.
The importance of demographics and subgroups in benchmarking
Demographic subgroups, such as age, tenure, tenancy type, and neighbourhood can identify pockets of high and low satisfaction. When these subgroups are analysed, particular business activities can be updated, introduced or ceased to align with areas of higher performance if applicable. Comparisons between benchmarking partners can also identify areas of best practice to share for particular segments such as residents in high-rise buildings, new tenancies, urban or rural areas, and young and elderly residents.
Other factors important in social housing benchmarking (and why)
In addition to benchmarking satisfaction and performance, it can be useful to benchmark cost data to ensure your asset management service is receiving the best value for money for repairs, cyclical maintenance, Decent Homes programmes and new build homes. You can also benchmark other transactions, for example income metrics like the number of days between rent due date and rent payment date. Cost benchmarking of all kinds is important to make sure you are squeezing value from every pound your organisation spends, and it’s also helpful to demonstrate to your residents that you actively undertake this type of exercise to reinforce trust in your fiscal responsibilities.
The difference between league tables and benchmarking
League tables are a ranking of performance metrics from worst to first. The concept of a social housing league table was put forward in the Social Housing Green Paper, with the sector asked to respond to related consultation questions to determine feasibility. But, publishing ranking tables of performance data for the public in a sector with little choice over which landlord you have, was seen to hold little value by the sector.
Another drawback of a national league table is the linear comparison of the wide range of organisations in terms of size, location, governance and stock make up – a direct comparison of all landlords with a single “top of the table” is not seen as fair or helpful to the sector. Providers prefer the benchmarking method of sharing performance data as a group and learning from the best as a more cooperative (less competitive) form of continuous service improvement.
Transactional vs Perception Benchmarking
Benchmarking can take place amongst a wide range of performance indicators, and those within housing are primarily focused on satisfaction. The timing of such satisfaction surveys is critical, as a respondent’s reply will likely be associated with their most recent contact with the landlord. For this reason, it’s important to understand if you will be benchmarking satisfaction as a perception survey – contacting a sample from your entire population in a truly random way, or if you are contacting residents immediately after a transaction, such as a payment or repair.
Because transactional satisfaction can be tied to a particular service, the feedback you receive and the insights you gain can lead directly to service improvements. This is highly valuable to individual organisations, but can’t always be benchmarked with reliability due to differences in question and response code wording, time delays between transactions and surveys, and frequency of data submissions to your benchmarking group.
Perception surveys are asked more widely amongst a population because they are not tied to a transaction. Perception surveys typically take place once per year, but they can also be carried out monthly. The submission of benchmarking data is controlled between certain dates and reporting windows. Perception surveys can be an indicator of a resident’s trust and confidence in your organisation, but feelings of general dissatisfaction typically cannot be tied directly to a service. Another drawback is the time lag between the survey and your benchmarked results can mean that service improvements that are needed have gone unresolved for months.