Quality Control: How to check for errors in your investment performance

Sean P. Gilligan
Author
May 5, 2021
15 min
Quality Control: How to check for errors in your investment performance

Recent investment performance calculation mistakes at Pennsylvania Public School Employees’ Retirement System (“PSERS”) have highlighted the importance of quality control reviews and raises questions about where risk exists, how these risks can be mitigated, and what role independent verifications should play in the quality control process.

What happened at PSERS?1

An error in the return calculation for Pennsylvania’s $64 billion state public school employee retirement plan has had serious implications for its beneficiaries and those involved in the calculation mistake.

In 2010, the plan, which was already underfunded, entered into a risk-sharing agreement where employees hired after 2011 would pay more into the plan if the return (average time-weighted return) over a specific time period fell below the actuarial value of asset (AVA) return of 6.36%.

In December 2020, the board announced that the plan had achieved a return of 6.38%, a mere 2 basis points above the minimum threshold. But in March the board changed its tune, announcing that the calculation was incorrect and the 100,000 or so employees hired since 2011 (and their employers) should have actually paid more into the plan.

What’s worse is PSERS also announced that the FBI is investigating the organization, although details of the probe have not yet been released.

According to PSERS, a consultant, that had calculated the return, came forward and admitted to the calculation error. But the board also said that it is looking into potential cover up by its staff. From what we know, at least 3 independent consultants were involved in providing data used for the calculations, calculating the returns, and verifying the returns. So, with all these experts involved, how could this happen and what can your firm do to avoid a similar situation?

Key issues to address in an investment performance quality control process

Firms should develop sound quality control processes to help identify errors before results are published. Often these processes either do not exist or are insufficient to identify issues. Following a robust quality control process that considers the key risks involved and then finds ways to mitigate these risks greatly increases the accuracy of presented investment performance.

Although we do not yet know the cause of the errors found in the PSERS case, we can highlight a few primary reasons errors occur in investment performance reporting. Primarily, errors found in published performance results are caused by:

  • Key Issue # 1 – Issues in the underlying data (e.g., incorrect or missing prices, unreconciled data, missing transactions, misclassified expenses, or failing to accrue fixed income)
  • Key Issue #2 – Mistakes in calculations (e.g., manual calculations that fail to match the intended methodology)
  • Key Issue #3 – Errors in reporting (e.g., publishing numbers that do not match the calculated results)

A robust quality control process should specifically address all three of these areas.

Considerations when designing a robust quality control process

Key Issue #1 – Issues in the underlying data

As they say, garbage in, garbage out. It is important to ask and address questions confirming the validity of data before it is used to calculate performance. Specifically, consider how the data used in the calculations is gathered, prepared, and reconciled before completing the calculations. Is there any formal signoff from the operations team confirming that the data is ready for use? Has a review of the data been conducted by an operations manager prior to this confirmation being made?

While deadlines to get performance published can be tight, taking the time to ensure that the underlying data is final and ready to use before performance is calculated can prevent headaches later on.

The following is a list of issues to look for when testing data validity:

  • Outlier performance – Portfolios performing differently than their peers may indicate a data issue or that the portfolio is mislabeled (i.e., tagged to a different strategy than it is invested in).
  • Differences between ending and beginning market values – Generally, we expect a portfolio’s market value at the end of one month and the beginning of the next month to be equal (unless using a system where external cashflows are recorded between months and differences like this are expected). Flagging differences can help identify data issues.
  • Offsetting decrease/increase in market value – Market values that suddenly increase or decrease and then return to the original value may have an incorrect price or transaction that should be researched.
  • Gaps in performance – A portfolio whose performance suddenly stops and then restarts may have missing data.
  • 0% returns – The portfolio may have liquidated and may no longer be under the firm’s discretionary management.
  • Very low market values – The portfolio may have closed and is only holding a small residual balance, which should be excluded from the firm’s discretionary management.
  • Net-of-fee returns higher than gross-of-fee returns – Seeing net returns that are higher than gross returns could indicate a data issue unless there are fee reversals you are aware of (e.g., performance fee accruals where previously accrued fees are adjusted back down).
  • Gross-of-fee returns and net of-fee returns are equal – If gross-of-fee and net-of-fee returns are always equal for a fee-paying portfolio, it is likely that the management fees are paid from an outside source (paid by check or out of a different portfolio). The returns labelled as net-of-fee in a case like this should be treated as gross-of-fee returns.

Key Issue #2 – Mistakes in calculations

Mistakes happen, but there are ways to reduce their frequency and impact. First, you’ll want to consider how manual your performance calculations are as well as the experience of the person completing the calculations.

Let’s face it, Excel is probably the most widely used tool in performance measurement, especially for smaller firms. While many firms likely find Excel to be a user-friendly tool for calculating performance statistics, it has its limitations. Studies have shown that up to 90% of spreadsheets contain errors and spreadsheets with lots of formulas are even more likely to contain mistakes. Whether it’s not properly dragging down a formula or referencing the wrong cell, fundamentally, the biggest problem is that users do not check their work or have carefully outlined procedures for confirming accuracy.

Although this may seem obvious, having a second set of eyes on a spreadsheet can save you from the embarrassing headache of having to explain errors in performance calculations. It is even better if this review is a multi-layered process. Having someone review details as well as someone to do a high-level “gut-check” to make sure the calculations and results make sense can reduce this risk. Depending on the size of your firm, this may be easier to accomplish with a third-party consultant, where you serve as a final layer of review.

Having this final “gut-check” can help prevent avoidable errors prior to publication. We find that this final “gut-check” is best performed by someone who knows the strategy intimately rather than a performance or compliance analyst, as these individuals may be too focused on the calculation details to take a step back and consider whether the returns make sense for the strategy and are in line with expectations.

If you use software to calculate performance, you can significantly reduce the risk of manual error, but due diligence should still be performed from time to time to manually prove out the accuracy of the calculations completed in the program. This does not need to be done every time but should be conducted when introducing a new software system and any time changes are made to the program.

Key Issue #3 – Errors in reporting

It may seem silly, but many performance reporting errors come from transposing strategy and benchmark returns in presentations or placing the return of one strategy in the factsheet of another. Therefore, it is important to consider how the final performance figures make it from the system or spreadsheet into the performance presentations. Are they typed? Copy and pasted? Or are the performance reports generated directly out of a system? It’s not enough to complete the calculations correctly, the final reports must also be accurate, so adding a step to review this is crucial.

A similar review process to the one described above can really make a difference, but ultimately, understanding the vulnerabilities of your performance reporting will help you design quality control procedures that address any exposure.

Calculations completed by external performance consultants

Whether performance is calculated internally or by a third-party performance consultant, the same key issues should be considered when designing the quality control process. Due diligence should be done on the performance consulting firm to evaluate the level of experience the firm has with calculating investment performance and what kind of quality control process they follow prior to providing results to your firm. This information will help you determine what reliance you can place on their procedures and what your firm should still check internally.

For example, outsourcing performance calculations to an individual or single-person firm likely necessitates a more in-depth review since this individual would not have the ability to have a second set of eyes on the results prior to providing them to your firm. However, even larger performance consulting firms with robust quality control processes may not have intimate knowledge of your strategies, meaning that, at a minimum, a final “gut-check” should be done by your firm prior to publication.

Reliance on independent performance verification firms to find errors

Many firms that hire performance verification firms rely on their verifier to be their quality control check; however, this may not be a good practice for a variety of reasons. If this is a common practice at your firm, you may want to check the scope of your engagement before relying too heavily on your verifier to find errors.

Verification is common for firms that claim compliance with the Global Investment Performance Standards (GIPS®). But even firms that claim compliance with the GIPS standards and receive a firm-wide verification are required to disclose that, “…Verification does not provide assurance on the accuracy of any specific performance report.

This is because verifiers are primarily focused on the existence and implementation of policies and procedures. While their review may help identify errors that exist in the sample selected for testing, it specifically does not certify the accuracy of presented results. While the verification process is valuable and often does turn up errors that need to be corrected, regardless of the scope of your engagement, a robust internal quality control process is likely still warranted.

Firms that are not GIPS compliant may engage verification firms for various types of attestation or review engagements like strategy exams or other non-GIPS performance reviews. In these situations, the scope of the engagement may be customized to meet the needs (and budget) of the firm seeking verification. A clear understanding of exactly what is in-scope and specifically what the verifier is opining on when issuing their report is key.

Situations where the engagement entails a detailed attestation tracing input data back to independent sources, confirming that calculations are carried out consistently, and verifying that published results match the calculations, allow for heavy reliance on the verifier as part of your quality control process.

Alternatively, when the scope merely consists of a high-level review confirming the appropriateness of the calculation methodology, a much more robust internal quality control process should be applied.

Knowing the scope of the engagement your firm has established with the verification firm is an important element in determining how much reliance can put on their review and findings, which can then be incorporated into the design of your own internal quality control procedures.

Key take-aways

Mistakes happen in investment performance reporting, but a robust quality control process can greatly mitigate this risk. Understanding the risks that exist, designing processes to test these risk areas, and understanding the role and engagement scope of all consultants involved are essential items in designing a quality control procedure that work for your firm – and hopefully one that will help you avoid situations like what happened with PSERS.

If you are not sure where to begin, we have tools and services available to help. Longs Peak uses proprietary software to calculate and analyze performance. Our software helps flag possible data issues and outlier performers and also produces performance reports directly from our performance system.

In addition, our performance consultants are available to work with your team to help identify potential vulnerabilities in your performance reporting process and can help you develop better quality control procedures, where needed.

Questions?

If you would like to learn more about our quality control process or any of the services we offer (like data and outlier testing) to help improve the accuracy and reliability of investment performance, contact us or email Sean Gilligan directly at sean@longspeakadvisory.com.

1 For more information on PSERS, please see this article from the Philadelphia Inquirer.

Recommended Post

View All Articles
ColoradoBiz Names Longs Peak’s Jocelyn Gilligan, CFA, CIPM as a GenZYZ Top Young Professional
Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine. As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”
March 14, 2023
15 min

Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine.

As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”

Jocelyn grew up in Boulder, CO and graduated from the University of Colorado. She started her career at Ernst & Young in New York City where she worked on their Financial Services Transfer Pricing Team. She transferred with EY to their office in Shanghai and then eventually to Hong Kong. Jocelyn left EY as a Manager and relocated back to Colorado where she and her husband started a family. Soon thereafter, Jocelyn and Sean founded Longs Peak out of a small one-car garage in their home in Longmont, CO. Now running a thriving team of 14, Jocelyn has weathered the ups and downs of entrepreneurship. She credits a lot of their success to their amazing team and the community of entrepreneurs they live near and network with (Longs Peak is an active member of EO (Entrepreneurs Organization)).

Jocelyn is a voting member of the PTO at her children’s school and a member of Women in Investment Performance Measurement, a group recently founded to support women in the investment performance industry.

Please join us in celebrating this year’s ColoradoBiz Top Young Professionals nominees. You can view the complete list of nominees here

About ColoradoBiz’s Top 25 Young Professionals

The 13th annual Gen XYZ awards is open to those under 40 who live and work in Colorado — numbered in the hundreds, making for difficult decisions and conversations among judges, as always. Applications were judged by our editorial board based on career achievement, community engagement and their stories of how they got to where they are now.

About Longs Peak

Longs Peak is a purpose and values-driven company. It is our mission to make investment performance information more transparent and reliable—empowering investors to make better, more informed investment decisions.

At the onset, we were looking to help smaller investment managers by giving them access to professional performance experts and tools typically only available to very large firms. We know that our work enables emerging managers to compete with the big guys and helps facilitate their growth. We strive to be our clients’ most valued outsource partner and to be known for our exceptional client service. We know that providing exceptional client service means that we must first create a culture that lives by the ideals we are trying to create for our clients. A place where incredibly talented individuals are empowered to put their best work into the hands of clients that truly value what we do. As a firm, we recognize that our greatest asset is people – both those we work with and those we work for. We continue to evolve into something that represents the needs of both of these groups and hope someday a GIPS Report is provided to every prospective investor in the world.

SEC Clarifies Marketing Rule: Gross-of-Fee Returns Allowed Under Certain Conditions
The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.
March 27, 2025
15 min

The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.

Extracted Performance: Gross Returns Can Stand Alone Under Specific Criteria

Investment advisers often present the performance of a single investment or a subset of a portfolio (“extracted performance”) in marketing materials. Historically, the SEC required both gross and net performance to be shown for such extracts. The new guidance provides a pathway for firms to display only gross-of-fee extracted performance, provided the following conditions are met:

  1. The extracted performance must be clearly identified as gross performance.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the extracted performance.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the extracted performance.

If these conditions are satisfied, the SEC staff has indicated they will not recommend enforcement action, even if the extracted performance is presented without corresponding net returns. This is a notable shift, as it allows firms to avoid the complex and often impractical task of allocating fees at the investment or sector level.

Portfolio and Investment Characteristics: Net-of-Fee Not Always Required

Another common industry question has been whether certain portfolio or investment characteristics—such as yield, volatility, Sharpe ratio, sector returns, or attribution analysis—constitute “performance” under the marketing rule, and if so, whether they must be presented net of fees.

The SEC’s latest guidance acknowledges that calculating these characteristics net of fees can be difficult and, in some cases, may lead to misleading results. As a result, the staff has confirmed that firms may present gross characteristics alone, without net characteristics, if they meet the following criteria:

  1. The characteristic must be clearly identified as calculated without the deduction of fees and expenses.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the gross characteristic.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the characteristic being presented.

As with extracted performance, these conditions help ensure that the presentation is not misleading, reducing the risk of enforcement action.

Bottom Line: A Practical Path Forward

This updated SEC guidance provides much-needed flexibility for investment managers, allowing for the presentation of gross-of-fee returns in a compliant manner. Firms that clearly disclose their approach and follow the specified conditions can reduce compliance burdens while still meeting investor protection standards. While this does not eliminate all complexities of the Marketing Rule, it does offer a practical solution that allows for more straightforward and meaningful performance reporting.

For firms navigating these changes, ensuring clear disclosures and maintaining compliance with the general prohibitions of the rule remains critical. Those who align their advertising materials with these guidelines can now confidently use gross-of-fee performance in a way that is both transparent and in compliance with regulatory requirements.

Questions?

If you have questions about calculating or presenting investment performance in a manner that complies with regulatory requirements or industry best practices, we would love to talk to you. Please feel free to email us at hello@longspeakadvisory.com.

New GIPS Standards Guidance for OCIOs: What You Need to Know
The Global Investment Performance Standards (GIPS®) have released a new Guidance Statement for OCIO Portfolios, bringing greater transparency and consistency to the way Outsourced Chief Investment Officers (OCIOs) report performance. This update is a significant milestone for firms managing OCIO Portfolios and asset owners looking to evaluate their OCIO providers.
February 3, 2025
15 min

The Global Investment Performance Standards (GIPS®) have released a new Guidance Statement for OCIO Portfolios, bringing greater transparency and consistency to the way Outsourced Chief Investment Officers (OCIOs) report performance. This update is a significant milestone for firms managing OCIO Portfolios and asset owners looking to evaluate their OCIO providers.

What is an OCIO?

An Outsourced Chief Investment Officer (OCIO) is a third-party fiduciary that provides both strategic investment advice and investment management services to institutional investors such as pension funds, endowments, and foundations. Instead of building an in-house investment team, asset owners delegate investment decisions to an OCIO, which handles everything from strategic planning to portfolio management.

Who Does the New Guidance Apply To?

The Guidance Statement for OCIO Portfolios applies when a firm provides both:

  1. Strategic investment advice, including developing or assessing an asset owner’s strategic asset allocation and investment policy statement.
  2. Investment management services, such as portfolio construction, fund and manager selection, and ongoing management.

This ensures that firms managing OCIO Portfolios follow standardized performance reporting, making it easier for prospective clients to compare OCIO providers.

Who is Exempt from the OCIO Guidance?

The guidance does not apply in the following scenarios:

  • Investment management without strategic advice – If a firm only manages investments without advising on asset allocation or investment policy.
  • Strategic advice without investment management – If a firm provides recommendations but does not manage the portfolio.
  • Partial OCIO portfolios – If a firm only manages a portion of a portfolio, rather than the full OCIO mandate.
  • Retail client portfolios – The guidance is specific to institutional OCIO Portfolios and does not apply to retail investors including larger wealth management portfolios.

Key Change: Required OCIO Composites

Previously, OCIO firms had flexibility in defining their performance composites. Now, the GIPS Standards introduce Required OCIO Composites, which categorize portfolios based on strategic asset allocation.

Types of Required OCIO Composites

  1. Liability-Focused Composites – Designed for portfolios aiming to meet specific liability streams, such as corporate pensions.
  2. Total Return Composites – Focused on capital appreciation, commonly used by endowments and foundations.

Firms must classify OCIO Portfolios based on their strategic allocation, not short-term tactical shifts. This standardization enhances comparability across OCIO providers. The specific allocation ranges for the required composites are as follows:

Required OCIO Composites for OCIO Portfolios

Required OCIO Composites
Source: Guidance Statement for OCIO Portfolios

Performance Calculation & Reporting

To ensure transparency, firms must follow specific rules for return calculations and fee disclosures:

  • Time-weighted returns (TWR) are required, even for portfolios with private equity or real estate holdings.
  • Both gross and net-of-fee returns must be presented to clarify the true cost of OCIO management.
  • Fee schedule disclosures must include all investment management fees, including fees from proprietary funds and third-party placements.

Enhanced Transparency in GIPS Reports

The new guidance also requires OCIO firms to disclose additional portfolio details, such as:

  • Annual asset allocation breakdowns (e.g., growth vs. liability-hedging assets).
  • Private market investment and hedge fund exposures.
  • Portfolio characteristics, such as funding ratios and duration for liability-focused portfolios.

By providing these details, OCIO firms enable prospective clients to make better-informed decisions when selecting an investment partner.

When Do These Changes Take Effect?

The Guidance Statement for OCIO Portfolios is effective December 31, 2025. From this date forward, GIPS Reports for Required OCIO Composites must follow the new standards. However, firms are encouraged to adopt the guidance earlier to improve transparency and reporting consistency.

Why This Matters

With OCIO services growing in popularity, this new guidance ensures that firms adhere to best practices in performance reporting. By establishing clear rules for composite classification, return calculation, and fee disclosure, the guidance empowers asset owners to compare OCIO providers with confidence.

As the December 31, 2025 deadline approaches, OCIO firms should begin aligning their reporting practices with this new guidance to stay ahead of the curve.

Don’t miss CFA Institute’s webinar scheduled for this Thursday February 6, 2025 to hear more on this guidance statement.

Questions?

If you have questions about the Guidance Statement for OCIO Portfolios or the Standards in general, we would love to talk to you. Longs Peak’s professionals have extensive experience helping firms become GIPS compliant as well as helping firms maintain their compliance with the GIPS Standards on an ongoing basis. Please feel free to email us at hello@longspeakadvisory.com.