Investment Performance Outlier Testing

Sean P. Gilligan
Author
January 29, 2020
15 min
Investment Performance Outlier Testing

For any firm that aggregates portfolios of the same strategy into a composite, or otherwise groups portfolios by mandate, how do you know that each portfolio truly follows that strategy? The answer is outlier testing.

Why Utilize Composites?

The GIPS standards require firms managing separate accounts to construct composites, which aggregate all discretionary portfolios of the same strategy. However, even for firms that are not GIPS compliant, the use of composites is considered best practice when reporting investment performance to prospective clients. Composites offer a more complete picture than presenting performance of a model or “representative portfolio” – which usually leave prospects wondering whether the information is truly representative or if the portfolio presented was “cherry picked.”

When creating and maintaining composites, firms must ensure that portfolios are included in the correct composite for the right time period – the period for which you had full discretion to implement the composite strategy for that portfolio. This is achieved by following a clearly documented set of policies and procedures for composite inclusion and exclusion. However, what happens when changes are made to a portfolio and those changes are not communicated to the person maintaining the composite?

In an ideal world, information in your firm would flow perfectly so that the person maintaining your composites knows exactly what is happening with the firm’s clients. In reality, client requests commonly result in small or temporary changes to the portfolio (e.g., halt trading, raise cash) that are not formally documented in the client’s investment guidelines or investment policy statement.

Without formal documentation of these changes, information may not flow down to the manager of your composites. While these minor or temporary changes may not affect the client’s long-term objectives, they may cause the portfolio to deviate from the strategy, requiring (at least temporary) removal from its composite. When these restricted portfolios are left in the composite, they often become performance outliers and create “noise” in the composite results. This “noise” prevents the composite from providing a meaningful representation of the portfolio manager’s ability to implement the strategy. This will also interfere with your prospective clients’ ability to analyze and interpret your performance results.

Why test for performance outliers?

Testing for performance outliers prior to finalizing and publishing performance results can help your firm remove this “noise” and can prevent costly errors in performance presentations. Firms that lack adequate composite construction policies and controls to ensure the policies are consistently followed often end up with errors in their composite presentations. In fact, it is very likely that errors in your performance exist. It is rare for us at Longs Peak to conduct an outlier analysis where no issues are found. Outlier testing should be completed quarterly and at a minimum, before any related verification or performance examination.

Many firms, especially those that are GIPS compliant, rely on their verifier to catch errors in their composites. We do not recommend this and suggest firms perform testing internally (or with the help of a performance consultant like Longs Peak) because:

  1. Verifiers only test a sample and will likely not catch all of your issues.
  2. Verification may happen months after the performance has been published. When errors are found, it may require redistribution of presentations with disclosures regarding prior performance errors.
  3. When verifiers find errors, they generally increase their sample size as well as their assessment of engagement risk. These two things lead to more time spent on the verification and a potential increase in your verification fee.

Even if not GIPS compliant, when firms use composites, regulators may test to ensure the composites are a meaningful representation of the strategy. In addition to improving accuracy, testing for performance outliers can help your firm‘s composites meet the standards expected by regulators.

How can performance outliers be identified?

Testing for performance outliers involves reviewing the performance of portfolios within the same composite or strategy to test if they are performing similarly. This testing allows you to flag any portfolios that may be performing differently so you can evaluate if their inclusion in the composite is appropriate.

For example, if your firm has a Large Cap Growth composite, testing performance outliers would involve compiling the return data for all of your Large Cap Growth portfolios, identifying which portfolios performed materially different from their peers, researching why they performed differently, and then taking the appropriate action if an issue is discovered. This may sound like a daunting task, but it doesn’t have to be. Let us walk you through this in more detail.

Some firms simply look at the absolute difference between each portfolio’s monthly return and the monthly return of the composite. While this may be straight forward, relying only on the absolute difference to determine outliers does not take into consideration the size of the return and the normal distribution of portfolio returns in the composite. For example, if you set a threshold to look at all portfolios that deviate from the composite return by 50bps, the result for a composite with low dispersion and a total return of 2% would very be different than a composite with higher dispersion and a total return of 20%.

In the outlier analysis Longs Peak conducts for clients, we use standard deviation in conjunction with a comparison of the absolute differences to identify the outlier portfolios that require review. Utilizing standard deviation allows us to identify portfolios that are truly outside the normal distribution of returns for each period. For example, reviewing all portfolios that are more than 3 standard deviations from the composite mean will provide the portfolios outside the normal distribution of returns for that period, regardless of the size of the return or the level of dispersion in that composite.

What to consider when reviewing outlier performance

The severity of the outlier

The larger the outlier, the more likely it is that the portfolio has an issue that would require it to be removed from the composite. We typically start by looking at the most extreme outliers first. Generally, we look at portfolios with performance periods flagged with +/-3 standard deviations from the mean return for the period. By addressing these first (including removing them if it is determined they do not belong in the composite), we are able to re-run the outlier test to assess what outliers exist without these extreme cases disrupting the analysis.

Once these extreme outliers are addressed, we move on to review the portfolios that are +/-2 standard deviations and even +/-1.5 standard deviations, if needed. We keep reviewing accounts with returns closer and closer to the composite’s mean return until we are consistently confirming that the portfolios do in fact belong in the composite and errors are not being found.

Each firm will be different in how much they need to drill down to get to a point of comfort that no more errors exist. If your composite is managed strictly to a model, the outliers will be very clear and easy to identify. If each portfolio you manage is customized, more research is often needed to determine if the outlier performance is simply a result of the portfolio’s customization or if the portfolio was included in the wrong composite.

How often the portfolio is an outlier

Longs Peak’s performance outlier reports show a portfolio’s performance, the number of standard deviations it is from the mean each month, and the number of months the portfolio was an outlier throughout its history in that composite. Our reports also show whether there was a cash flow during that period or not. The following are examples of outlier frequencies we evaluate:

Infrequent: If you see that a portfolio is only an outlier for one month and that month had a large cash flow, then you will know that the portfolio is likely only an outlier for that period because of the cash flow and, often, no further research is required.

Frequent: If you can see that the portfolio is an outlier for most of the months under review, then you will know that there is likely an issue with this portfolio.

As of a specific date: If you can see that the portfolio was not an outlier historically, but became a frequent outlier from a certain month forward, this may indicate that a restriction was added or that the strategy changed as of that period. The portfolio may then need to be reclassified to the appropriate composite or flagged as non-discretionary.

The most common causes of outlier performance and how to address performance outliers

Common causes of outlier performance:

  • Data issues – When outliers are extreme, it is likely that there is an issue with the data. Examples include a pricing issue that caused a material jump in performance or a late dividend hitting a portfolio that is closing and had most of its assets already transferred out. These issues are often easily addressed, depending on the circumstance of each case.
  • Cash flows – If a portfolio is only an outlier for one month and during that month the portfolio experienced a large cash flow, this is likely the reason for the outlier performance. If the portfolio had high cash for a period of time around the cash flow and the market moved during that period, this portfolio likely would perform differently than its fully invested peers. Nothing needs to be done in this scenario since the outlier performance is explained and there is no indication that the portfolio is invested incorrectly or grouped with the wrong portfolios.
  • Legacy positions or other client restrictions – If your clients hold legacy positions that you are restricted from selling or have other similar restrictions, this will likely cause these portfolios to perform differently when compared to their unrestricted peers. Depending on your composite construction rules, unless immaterial, these portfolios likely need to be excluded from the composite. With these portfolios removed, other outliers may appear that were not as noticeable when the restricted portfolios were included. It is important to refer to your firm’s composite construction policies, which should outline clear parameters for when restricted portfolios should be included/excluded in composites.
  • Portfolio categorized incorrectly – A portfolio may appear as an outlier because it was placed in the wrong composite. This often happens if a portfolio’s composite changed and it was not removed from its prior composite. If this is the case, the portfolio must be removed (after the change) and added to the new composite based on the timing outlined in your firm’s composite construction policies.
  • Portfolio managed incorrectly – Performance outlier analysis may help identify a portfolio that is managed to the wrong strategy. For example, it is possible that the portfolio is grouped with the correct portfolios, but the wrong strategy was implemented in the portfolio. This is one of the most important errors that performance outlier testing can identify because it means that the client is actually not having their money managed to the strategy for which your firm was hired. In this case, the portfolio would need to be rebalanced to the correct strategy. Likely, a review of the history would need to be conducted as well to ensure the client was not disadvantaged by the error.
  • High dispersion between portfolio managers – Especially when more than one portfolio manager is implementing the same composite at your firm, material differences may exist in the way they each manage the strategy. Outlier performers may be due to differences in the portfolio managers’ discretionary management. If the composite is being sold as one cohesive product, it is important to identify where the portfolio managers deviate and determine if they can work more closely together to avoid high dispersion or if the strategy should actually be run as two different products.

When researching outlier performance, keep in mind that, on its own, a portfolio’s performance deviating from its peers is not a valid reason to remove the portfolio from its composite. You need to determine the root cause of the deviation and remove the portfolio from its composite only if the root cause was client-driven. If the deviation was caused by tactical, discretionary moves made by the portfolio manager, the portfolio must remain in the composite as its performance is still a representation of the portfolio manager’s implementation of the strategy.

Ready to implement performance outlier testing at your firm?

While it is best practice to create a flow of information that will allow portfolios to proactively be included/excluded in the correct composite at the appropriate time, testing for performance outliers acts as a back-up plan to catch anything that was missed.

If analyzing your composite data to identify performance outliers is not something you have the resources to do internally, Longs Peak is available to help. Longs Peak offers both consulting and reporting services that can assist your firm with outlier analysis. Conducting outlier analysis should be done at least quarterly to help ensure your firm is managing your portfolios consistently and are reporting strategy or composite performance that is meaningful and accurate. Please contact us to discuss how we can help implement this practice for your firm.

Questions? 

If you have questions about investment performance, composite construction, or the GIPS standards, we would be love to talk to you. Longs Peak’s professionals have extensive experience helping firms with all of their investment performance needs. Please feel free to email Sean Gilligan directly at sean@longspeakadvisory.com.

Recommended Post

View All Articles
ColoradoBiz Names Longs Peak’s Jocelyn Gilligan, CFA, CIPM as a GenZYZ Top Young Professional
Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine. As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”
March 14, 2023
15 min

Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine.

As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”

Jocelyn grew up in Boulder, CO and graduated from the University of Colorado. She started her career at Ernst & Young in New York City where she worked on their Financial Services Transfer Pricing Team. She transferred with EY to their office in Shanghai and then eventually to Hong Kong. Jocelyn left EY as a Manager and relocated back to Colorado where she and her husband started a family. Soon thereafter, Jocelyn and Sean founded Longs Peak out of a small one-car garage in their home in Longmont, CO. Now running a thriving team of 14, Jocelyn has weathered the ups and downs of entrepreneurship. She credits a lot of their success to their amazing team and the community of entrepreneurs they live near and network with (Longs Peak is an active member of EO (Entrepreneurs Organization)).

Jocelyn is a voting member of the PTO at her children’s school and a member of Women in Investment Performance Measurement, a group recently founded to support women in the investment performance industry.

Please join us in celebrating this year’s ColoradoBiz Top Young Professionals nominees. You can view the complete list of nominees here

About ColoradoBiz’s Top 25 Young Professionals

The 13th annual Gen XYZ awards is open to those under 40 who live and work in Colorado — numbered in the hundreds, making for difficult decisions and conversations among judges, as always. Applications were judged by our editorial board based on career achievement, community engagement and their stories of how they got to where they are now.

About Longs Peak

Longs Peak is a purpose and values-driven company. It is our mission to make investment performance information more transparent and reliable—empowering investors to make better, more informed investment decisions.

At the onset, we were looking to help smaller investment managers by giving them access to professional performance experts and tools typically only available to very large firms. We know that our work enables emerging managers to compete with the big guys and helps facilitate their growth. We strive to be our clients’ most valued outsource partner and to be known for our exceptional client service. We know that providing exceptional client service means that we must first create a culture that lives by the ideals we are trying to create for our clients. A place where incredibly talented individuals are empowered to put their best work into the hands of clients that truly value what we do. As a firm, we recognize that our greatest asset is people – both those we work with and those we work for. We continue to evolve into something that represents the needs of both of these groups and hope someday a GIPS Report is provided to every prospective investor in the world.

SEC Clarifies Marketing Rule: Gross-of-Fee Returns Allowed Under Certain Conditions
The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.
March 27, 2025
15 min

The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.

Extracted Performance: Gross Returns Can Stand Alone Under Specific Criteria

Investment advisers often present the performance of a single investment or a subset of a portfolio (“extracted performance”) in marketing materials. Historically, the SEC required both gross and net performance to be shown for such extracts. The new guidance provides a pathway for firms to display only gross-of-fee extracted performance, provided the following conditions are met:

  1. The extracted performance must be clearly identified as gross performance.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the extracted performance.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the extracted performance.

If these conditions are satisfied, the SEC staff has indicated they will not recommend enforcement action, even if the extracted performance is presented without corresponding net returns. This is a notable shift, as it allows firms to avoid the complex and often impractical task of allocating fees at the investment or sector level.

Portfolio and Investment Characteristics: Net-of-Fee Not Always Required

Another common industry question has been whether certain portfolio or investment characteristics—such as yield, volatility, Sharpe ratio, sector returns, or attribution analysis—constitute “performance” under the marketing rule, and if so, whether they must be presented net of fees.

The SEC’s latest guidance acknowledges that calculating these characteristics net of fees can be difficult and, in some cases, may lead to misleading results. As a result, the staff has confirmed that firms may present gross characteristics alone, without net characteristics, if they meet the following criteria:

  1. The characteristic must be clearly identified as calculated without the deduction of fees and expenses.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the gross characteristic.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the characteristic being presented.

As with extracted performance, these conditions help ensure that the presentation is not misleading, reducing the risk of enforcement action.

Bottom Line: A Practical Path Forward

This updated SEC guidance provides much-needed flexibility for investment managers, allowing for the presentation of gross-of-fee returns in a compliant manner. Firms that clearly disclose their approach and follow the specified conditions can reduce compliance burdens while still meeting investor protection standards. While this does not eliminate all complexities of the Marketing Rule, it does offer a practical solution that allows for more straightforward and meaningful performance reporting.

For firms navigating these changes, ensuring clear disclosures and maintaining compliance with the general prohibitions of the rule remains critical. Those who align their advertising materials with these guidelines can now confidently use gross-of-fee performance in a way that is both transparent and in compliance with regulatory requirements.

Questions?

If you have questions about calculating or presenting investment performance in a manner that complies with regulatory requirements or industry best practices, we would love to talk to you. Please feel free to email us at hello@longspeakadvisory.com.

New GIPS Standards Guidance for OCIOs: What You Need to Know
The Global Investment Performance Standards (GIPS®) have released a new Guidance Statement for OCIO Portfolios, bringing greater transparency and consistency to the way Outsourced Chief Investment Officers (OCIOs) report performance. This update is a significant milestone for firms managing OCIO Portfolios and asset owners looking to evaluate their OCIO providers.
February 3, 2025
15 min

The Global Investment Performance Standards (GIPS®) have released a new Guidance Statement for OCIO Portfolios, bringing greater transparency and consistency to the way Outsourced Chief Investment Officers (OCIOs) report performance. This update is a significant milestone for firms managing OCIO Portfolios and asset owners looking to evaluate their OCIO providers.

What is an OCIO?

An Outsourced Chief Investment Officer (OCIO) is a third-party fiduciary that provides both strategic investment advice and investment management services to institutional investors such as pension funds, endowments, and foundations. Instead of building an in-house investment team, asset owners delegate investment decisions to an OCIO, which handles everything from strategic planning to portfolio management.

Who Does the New Guidance Apply To?

The Guidance Statement for OCIO Portfolios applies when a firm provides both:

  1. Strategic investment advice, including developing or assessing an asset owner’s strategic asset allocation and investment policy statement.
  2. Investment management services, such as portfolio construction, fund and manager selection, and ongoing management.

This ensures that firms managing OCIO Portfolios follow standardized performance reporting, making it easier for prospective clients to compare OCIO providers.

Who is Exempt from the OCIO Guidance?

The guidance does not apply in the following scenarios:

  • Investment management without strategic advice – If a firm only manages investments without advising on asset allocation or investment policy.
  • Strategic advice without investment management – If a firm provides recommendations but does not manage the portfolio.
  • Partial OCIO portfolios – If a firm only manages a portion of a portfolio, rather than the full OCIO mandate.
  • Retail client portfolios – The guidance is specific to institutional OCIO Portfolios and does not apply to retail investors including larger wealth management portfolios.

Key Change: Required OCIO Composites

Previously, OCIO firms had flexibility in defining their performance composites. Now, the GIPS Standards introduce Required OCIO Composites, which categorize portfolios based on strategic asset allocation.

Types of Required OCIO Composites

  1. Liability-Focused Composites – Designed for portfolios aiming to meet specific liability streams, such as corporate pensions.
  2. Total Return Composites – Focused on capital appreciation, commonly used by endowments and foundations.

Firms must classify OCIO Portfolios based on their strategic allocation, not short-term tactical shifts. This standardization enhances comparability across OCIO providers. The specific allocation ranges for the required composites are as follows:

Required OCIO Composites for OCIO Portfolios

Required OCIO Composites
Source: Guidance Statement for OCIO Portfolios

Performance Calculation & Reporting

To ensure transparency, firms must follow specific rules for return calculations and fee disclosures:

  • Time-weighted returns (TWR) are required, even for portfolios with private equity or real estate holdings.
  • Both gross and net-of-fee returns must be presented to clarify the true cost of OCIO management.
  • Fee schedule disclosures must include all investment management fees, including fees from proprietary funds and third-party placements.

Enhanced Transparency in GIPS Reports

The new guidance also requires OCIO firms to disclose additional portfolio details, such as:

  • Annual asset allocation breakdowns (e.g., growth vs. liability-hedging assets).
  • Private market investment and hedge fund exposures.
  • Portfolio characteristics, such as funding ratios and duration for liability-focused portfolios.

By providing these details, OCIO firms enable prospective clients to make better-informed decisions when selecting an investment partner.

When Do These Changes Take Effect?

The Guidance Statement for OCIO Portfolios is effective December 31, 2025. From this date forward, GIPS Reports for Required OCIO Composites must follow the new standards. However, firms are encouraged to adopt the guidance earlier to improve transparency and reporting consistency.

Why This Matters

With OCIO services growing in popularity, this new guidance ensures that firms adhere to best practices in performance reporting. By establishing clear rules for composite classification, return calculation, and fee disclosure, the guidance empowers asset owners to compare OCIO providers with confidence.

As the December 31, 2025 deadline approaches, OCIO firms should begin aligning their reporting practices with this new guidance to stay ahead of the curve.

Don’t miss CFA Institute’s webinar scheduled for this Thursday February 6, 2025 to hear more on this guidance statement.

Questions?

If you have questions about the Guidance Statement for OCIO Portfolios or the Standards in general, we would love to talk to you. Longs Peak’s professionals have extensive experience helping firms become GIPS compliant as well as helping firms maintain their compliance with the GIPS Standards on an ongoing basis. Please feel free to email us at hello@longspeakadvisory.com.