Investment Performance Outlier Testing

Sean P. Gilligan
Author
January 29, 2020
15 min
Investment Performance Outlier Testing

For any firm that aggregates portfolios of the same strategy into a composite, or otherwise groups portfolios by mandate, how do you know that each portfolio truly follows that strategy? The answer is outlier testing.

Why Utilize Composites?

The GIPS standards require firms managing separate accounts to construct composites, which aggregate all discretionary portfolios of the same strategy. However, even for firms that are not GIPS compliant, the use of composites is considered best practice when reporting investment performance to prospective clients. Composites offer a more complete picture than presenting performance of a model or “representative portfolio” – which usually leave prospects wondering whether the information is truly representative or if the portfolio presented was “cherry picked.”

When creating and maintaining composites, firms must ensure that portfolios are included in the correct composite for the right time period – the period for which you had full discretion to implement the composite strategy for that portfolio. This is achieved by following a clearly documented set of policies and procedures for composite inclusion and exclusion. However, what happens when changes are made to a portfolio and those changes are not communicated to the person maintaining the composite?

In an ideal world, information in your firm would flow perfectly so that the person maintaining your composites knows exactly what is happening with the firm’s clients. In reality, client requests commonly result in small or temporary changes to the portfolio (e.g., halt trading, raise cash) that are not formally documented in the client’s investment guidelines or investment policy statement.

Without formal documentation of these changes, information may not flow down to the manager of your composites. While these minor or temporary changes may not affect the client’s long-term objectives, they may cause the portfolio to deviate from the strategy, requiring (at least temporary) removal from its composite. When these restricted portfolios are left in the composite, they often become performance outliers and create “noise” in the composite results. This “noise” prevents the composite from providing a meaningful representation of the portfolio manager’s ability to implement the strategy. This will also interfere with your prospective clients’ ability to analyze and interpret your performance results.

Why test for performance outliers?

Testing for performance outliers prior to finalizing and publishing performance results can help your firm remove this “noise” and can prevent costly errors in performance presentations. Firms that lack adequate composite construction policies and controls to ensure the policies are consistently followed often end up with errors in their composite presentations. In fact, it is very likely that errors in your performance exist. It is rare for us at Longs Peak to conduct an outlier analysis where no issues are found. Outlier testing should be completed quarterly and at a minimum, before any related verification or performance examination.

Many firms, especially those that are GIPS compliant, rely on their verifier to catch errors in their composites. We do not recommend this and suggest firms perform testing internally (or with the help of a performance consultant like Longs Peak) because:

  1. Verifiers only test a sample and will likely not catch all of your issues.
  2. Verification may happen months after the performance has been published. When errors are found, it may require redistribution of presentations with disclosures regarding prior performance errors.
  3. When verifiers find errors, they generally increase their sample size as well as their assessment of engagement risk. These two things lead to more time spent on the verification and a potential increase in your verification fee.

Even if not GIPS compliant, when firms use composites, regulators may test to ensure the composites are a meaningful representation of the strategy. In addition to improving accuracy, testing for performance outliers can help your firm‘s composites meet the standards expected by regulators.

How can performance outliers be identified?

Testing for performance outliers involves reviewing the performance of portfolios within the same composite or strategy to test if they are performing similarly. This testing allows you to flag any portfolios that may be performing differently so you can evaluate if their inclusion in the composite is appropriate.

For example, if your firm has a Large Cap Growth composite, testing performance outliers would involve compiling the return data for all of your Large Cap Growth portfolios, identifying which portfolios performed materially different from their peers, researching why they performed differently, and then taking the appropriate action if an issue is discovered. This may sound like a daunting task, but it doesn’t have to be. Let us walk you through this in more detail.

Some firms simply look at the absolute difference between each portfolio’s monthly return and the monthly return of the composite. While this may be straight forward, relying only on the absolute difference to determine outliers does not take into consideration the size of the return and the normal distribution of portfolio returns in the composite. For example, if you set a threshold to look at all portfolios that deviate from the composite return by 50bps, the result for a composite with low dispersion and a total return of 2% would very be different than a composite with higher dispersion and a total return of 20%.

In the outlier analysis Longs Peak conducts for clients, we use standard deviation in conjunction with a comparison of the absolute differences to identify the outlier portfolios that require review. Utilizing standard deviation allows us to identify portfolios that are truly outside the normal distribution of returns for each period. For example, reviewing all portfolios that are more than 3 standard deviations from the composite mean will provide the portfolios outside the normal distribution of returns for that period, regardless of the size of the return or the level of dispersion in that composite.

What to consider when reviewing outlier performance

The severity of the outlier

The larger the outlier, the more likely it is that the portfolio has an issue that would require it to be removed from the composite. We typically start by looking at the most extreme outliers first. Generally, we look at portfolios with performance periods flagged with +/-3 standard deviations from the mean return for the period. By addressing these first (including removing them if it is determined they do not belong in the composite), we are able to re-run the outlier test to assess what outliers exist without these extreme cases disrupting the analysis.

Once these extreme outliers are addressed, we move on to review the portfolios that are +/-2 standard deviations and even +/-1.5 standard deviations, if needed. We keep reviewing accounts with returns closer and closer to the composite’s mean return until we are consistently confirming that the portfolios do in fact belong in the composite and errors are not being found.

Each firm will be different in how much they need to drill down to get to a point of comfort that no more errors exist. If your composite is managed strictly to a model, the outliers will be very clear and easy to identify. If each portfolio you manage is customized, more research is often needed to determine if the outlier performance is simply a result of the portfolio’s customization or if the portfolio was included in the wrong composite.

How often the portfolio is an outlier

Longs Peak’s performance outlier reports show a portfolio’s performance, the number of standard deviations it is from the mean each month, and the number of months the portfolio was an outlier throughout its history in that composite. Our reports also show whether there was a cash flow during that period or not. The following are examples of outlier frequencies we evaluate:

Infrequent: If you see that a portfolio is only an outlier for one month and that month had a large cash flow, then you will know that the portfolio is likely only an outlier for that period because of the cash flow and, often, no further research is required.

Frequent: If you can see that the portfolio is an outlier for most of the months under review, then you will know that there is likely an issue with this portfolio.

As of a specific date: If you can see that the portfolio was not an outlier historically, but became a frequent outlier from a certain month forward, this may indicate that a restriction was added or that the strategy changed as of that period. The portfolio may then need to be reclassified to the appropriate composite or flagged as non-discretionary.

The most common causes of outlier performance and how to address performance outliers

Common causes of outlier performance:

  • Data issues – When outliers are extreme, it is likely that there is an issue with the data. Examples include a pricing issue that caused a material jump in performance or a late dividend hitting a portfolio that is closing and had most of its assets already transferred out. These issues are often easily addressed, depending on the circumstance of each case.
  • Cash flows – If a portfolio is only an outlier for one month and during that month the portfolio experienced a large cash flow, this is likely the reason for the outlier performance. If the portfolio had high cash for a period of time around the cash flow and the market moved during that period, this portfolio likely would perform differently than its fully invested peers. Nothing needs to be done in this scenario since the outlier performance is explained and there is no indication that the portfolio is invested incorrectly or grouped with the wrong portfolios.
  • Legacy positions or other client restrictions – If your clients hold legacy positions that you are restricted from selling or have other similar restrictions, this will likely cause these portfolios to perform differently when compared to their unrestricted peers. Depending on your composite construction rules, unless immaterial, these portfolios likely need to be excluded from the composite. With these portfolios removed, other outliers may appear that were not as noticeable when the restricted portfolios were included. It is important to refer to your firm’s composite construction policies, which should outline clear parameters for when restricted portfolios should be included/excluded in composites.
  • Portfolio categorized incorrectly – A portfolio may appear as an outlier because it was placed in the wrong composite. This often happens if a portfolio’s composite changed and it was not removed from its prior composite. If this is the case, the portfolio must be removed (after the change) and added to the new composite based on the timing outlined in your firm’s composite construction policies.
  • Portfolio managed incorrectly – Performance outlier analysis may help identify a portfolio that is managed to the wrong strategy. For example, it is possible that the portfolio is grouped with the correct portfolios, but the wrong strategy was implemented in the portfolio. This is one of the most important errors that performance outlier testing can identify because it means that the client is actually not having their money managed to the strategy for which your firm was hired. In this case, the portfolio would need to be rebalanced to the correct strategy. Likely, a review of the history would need to be conducted as well to ensure the client was not disadvantaged by the error.
  • High dispersion between portfolio managers – Especially when more than one portfolio manager is implementing the same composite at your firm, material differences may exist in the way they each manage the strategy. Outlier performers may be due to differences in the portfolio managers’ discretionary management. If the composite is being sold as one cohesive product, it is important to identify where the portfolio managers deviate and determine if they can work more closely together to avoid high dispersion or if the strategy should actually be run as two different products.

When researching outlier performance, keep in mind that, on its own, a portfolio’s performance deviating from its peers is not a valid reason to remove the portfolio from its composite. You need to determine the root cause of the deviation and remove the portfolio from its composite only if the root cause was client-driven. If the deviation was caused by tactical, discretionary moves made by the portfolio manager, the portfolio must remain in the composite as its performance is still a representation of the portfolio manager’s implementation of the strategy.

Ready to implement performance outlier testing at your firm?

While it is best practice to create a flow of information that will allow portfolios to proactively be included/excluded in the correct composite at the appropriate time, testing for performance outliers acts as a back-up plan to catch anything that was missed.

If analyzing your composite data to identify performance outliers is not something you have the resources to do internally, Longs Peak is available to help. Longs Peak offers both consulting and reporting services that can assist your firm with outlier analysis. Conducting outlier analysis should be done at least quarterly to help ensure your firm is managing your portfolios consistently and are reporting strategy or composite performance that is meaningful and accurate. Please contact us to discuss how we can help implement this practice for your firm.

Questions? 

If you have questions about investment performance, composite construction, or the GIPS standards, we would be love to talk to you. Longs Peak’s professionals have extensive experience helping firms with all of their investment performance needs. Please feel free to email Sean Gilligan directly at sean@longspeakadvisory.com.

Recommended Post

View All Articles
Key Takeaways from the 2025 PMAR Conference
This year’s PMAR Conference delivered timely and thought-provoking content for performance professionals across the industry. In this post, we’ve highlighted our top takeaways from the event—including a recap of the WiPM gathering.
May 29, 2025
15 min

The Performance Measurement, Attribution & Risk (PMAR) Conference is always a highlight for investment performance professionals—and this year’s event did not disappoint. With a packed agenda spanning everything from economic uncertainty and automation to evolving training needs and private market complexities, PMAR 2025 gave attendees plenty to think about.

Here are some of our key takeaways from this year’s event:

Women in Performance Measurement (WiPM)

Although not officially a part of PMAR, WiPM often schedules its annual in-person gathering during the same week to take advantage of the broader industry presence at the event. This year’s in-person gathering, united female professionals from across the country for a full day of connection, learning, and mentorship. The agenda struck a thoughtful balance between professional development and personal connection, with standout sessions on AI and machine learning, resume building, and insights from the WiPM mentoring program. A consistent favorite among attendees is the interactive format—discussions are engaging, and the support among members is truly energizing. The day concluded with a cocktail reception and dinner, reinforcing the group’s strong sense of community and its ongoing commitment to advancing women in the performance measurement profession.

If you’re not yet a member and are interested in joining the community, find WiPM here on LinkedIn.

Uncertainty, Not Risk, is Driving Market Volatility

John Longo, Ph.D., Rutgers Business School kicked off the conference with a deep dive into the global economy, and his message was clear: today’s markets are more uncertain than risky. Tariffs, political volatility, and unconventional strategies—like the idea of purchasing Greenland—are reshaping global trade and investment decisions. His suggestion? Investors may want to look beyond U.S. borders and consider assets like gold or emerging markets as a hedge.

Longo also highlighted the looming national debt problem and inflationary effects of protectionist policies. For performance professionals, the implication is clear: macro-level policy choices are creating noise that can obscure traditional risk metrics. Understanding the difference between risk and uncertainty is more important than ever.

The Future of Training: Customized, Continuous, and Collaborative

In the “Developing Staff for Success” session, Frances Barney, CFA (former head of investment performance and risk analysis for BNY Mellon) and our very own Jocelyn Gilligan, CFA, CIPM explored the evolving nature of training in our field. The key message: cookie-cutter training doesn't cut it anymore. With increasing regulatory complexity and rapidly advancing technology, firms must invest in flexible, personalized learning programs.

Whether it's improving communication skills, building tech proficiency, or embedding a culture of curiosity, the session emphasized that training must be more than a check-the-box activity. Ongoing mentorship, cross-training, and embracing neurodiversity in learning styles are all part of building high-performing, engaged teams.

AI is Here—But It Needs a Human Co-Pilot

Several sessions explored the growing role of AI and automation in performance and reporting. The consensus? AI holds immense promise, but without strong data governance and human oversight, it’s not a silver bullet. From hallucinations in generative models to the ethical challenges of data usage, AI introduces new risks even as it streamlines workflows.

Use cases presented ranged from anomaly detection and report generation to client communication enhancements and predictive exception handling. But again and again, speakers emphasized: AI should augment, not replace, human expertise.

Private Markets Require Purpose-Built Tools

Private equity, private credit, real estate, and hedge funds remain among the trickiest asset classes to measure. Whether debating IRR vs. TWR, handling data lags, or selecting appropriate benchmarks, this year's sessions highlighted just how much nuance is involved in getting private market reporting right.

One particularly compelling idea: using replicating portfolios of public assets to assess the risk and performance of illiquid investments. This approach offers more transparency and a better sense of underlying exposures, especially in the absence of timely valuations.

Shorting and Leverage Complicate Performance Attribution

Calculating performance in long/short portfolios isn’t straightforward—and using absolute values can create misleading results. A session on this topic broke down the mechanics of short selling and explained why contribution-based return attribution is essential for accurate reporting.

The key insight: portfolio-level returns can fall outside the range of individual asset returns, especially in leveraged portfolios. Understanding the directional nature of each position is crucial for both internal attribution and external communication.

The SEC is Watching—Are You Ready?

Compliance was another hot topic, especially in light of recent enforcement actions under the SEC Marketing Rule. From misuse of hypothetical performance to sloppy use of testimonials, the panelists shared hard-earned lessons and emphasized the importance of documentation. This panel was moderated by Longs Peak’s Matt Deatherage, CFA, CIPM and included Lance Dial, of K&L Gates along with Thayne Gould from Vigilant.

FAQs have helped clarify gray areas (especially around extracted performance and proximity of net vs. gross returns), but more guidance is expected—particularly on model fees and performance portability. If you're not already documenting every performance claim, now is the time to start.

“Phantom Alpha” Is Real—And Preventable

David Spaulding of TSG, closed the conference with a deep dive into benchmark construction and the potential for “phantom alpha.” Even small differences in rebalancing frequency between portfolios and their benchmarks can create misleading outperformance. His recommendation? Either sync your rebalancing schedules or clearly disclose the differences.

This session served as a great reminder that even small implementation details can significantly impact reported performance—and that transparency is essential to maintaining trust.

Final Thoughts

From automation to attribution, PMAR 2025 showcased the depth and complexity of our field. If there’s one overarching takeaway, it’s that while tools and techniques continue to evolve, the core principles—transparency, accuracy, and accountability—remain as important a sever.

Did you attend PMAR this year? We’d love to hear your biggest takeaways. Reach out to us at hello@longspeakadvisory.com or drop us a note on LinkedIn!

ColoradoBiz Names Longs Peak’s Jocelyn Gilligan, CFA, CIPM as a Gen XYZ Top Young Professional
Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine. As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”
March 14, 2023
15 min

Longs Peak is pleased to announce that Partner and Co-Founder, Jocelyn Gilligan has been named a GenXYZ Top Young Professional by ColoradoBiz Magazine.

As ColoradoBiz states, “They’re uncommon achievers, whether as entrepreneurs, CEOs, nonprofit leaders, visionaries critical to their companies’ success or, in some cases, all of those roles. This year’s Top 25 Young Professionals figure to continue making a difference professionally and in their communities for years to come.”

Jocelyn grew up in Boulder, CO and graduated from the University of Colorado. She started her career at Ernst & Young in New York City where she worked on their Financial Services Transfer Pricing Team. She transferred with EY to their office in Shanghai and then eventually to Hong Kong. Jocelyn left EY as a Manager and relocated back to Colorado where she and her husband started a family. Soon thereafter, Jocelyn and Sean founded Longs Peak out of a small one-car garage in their home in Longmont, CO. Now running a thriving team of 14, Jocelyn has weathered the ups and downs of entrepreneurship. She credits a lot of their success to their amazing team and the community of entrepreneurs they live near and network with (Longs Peak is an active member of EO (Entrepreneurs Organization)).

Jocelyn is a voting member of the PTO at her children’s school and a member of Women in Investment Performance Measurement, a group recently founded to support women in the investment performance industry.

About ColoradoBiz’s Top 25 Young Professionals

The 13th annual Gen XYZ awards is open to those under 40 who live and work in Colorado — numbered in the hundreds, making for difficult decisions and conversations among judges, as always. Applications were judged by our editorial board based on career achievement, community engagement and their stories of how they got to where they are now.

About Longs Peak

Longs Peak is a purpose and values-driven company. It is our mission to make investment performance information more transparent and reliable—empowering investors to make better, more informed investment decisions.

At the onset, we were looking to help smaller investment managers by giving them access to professional performance experts and tools typically only available to very large firms. We know that our work enables emerging managers to compete with the big guys and helps facilitate their growth. We strive to be our clients’ most valued outsource partner and to be known for our exceptional client service. We know that providing exceptional client service means that we must first create a culture that lives by the ideals we are trying to create for our clients. A place where incredibly talented individuals are empowered to put their best work into the hands of clients that truly value what we do. As a firm, we recognize that our greatest asset is people – both those we work with and those we work for. We continue to evolve into something that represents the needs of both of these groups and hope someday a GIPS Report is provided to every prospective investor in the world.

SEC Clarifies Marketing Rule: Gross-of-Fee Returns Allowed Under Certain Conditions
The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.
March 27, 2025
15 min

The investment management industry has spent significant time grappling with the SEC’s Marketing Rule and the question of whether gross-of-fee returns can be presented without corresponding net-of-fee returns in certain cases. Many firms have invested resources in trying to allocate fees to individual securities and sectors in an effort to comply. However, the SEC has now issued two FAQs (March 19, 2025) that provide much appreciated clarity on extracted performance and portfolio characteristics. The key takeaway? It is possible to present gross-of-fee returns without net-of-fee returns—if certain conditions are met.

Extracted Performance: Gross Returns Can Stand Alone Under Specific Criteria

Investment advisers often present the performance of a single investment or a subset of a portfolio (“extracted performance”) in marketing materials. Historically, the SEC required both gross and net performance to be shown for such extracts. The new guidance provides a pathway for firms to display only gross-of-fee extracted performance, provided the following conditions are met:

  1. The extracted performance must be clearly identified as gross performance.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the extracted performance.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the extracted performance.

If these conditions are satisfied, the SEC staff has indicated they will not recommend enforcement action, even if the extracted performance is presented without corresponding net returns. This is a notable shift, as it allows firms to avoid the complex and often impractical task of allocating fees at the investment or sector level.

Portfolio and Investment Characteristics: Net-of-Fee Not Always Required

Another common industry question has been whether certain portfolio or investment characteristics—such as yield, volatility, Sharpe ratio, sector returns, or attribution analysis—constitute “performance” under the marketing rule, and if so, whether they must be presented net of fees.

The SEC’s latest guidance acknowledges that calculating these characteristics net of fees can be difficult and, in some cases, may lead to misleading results. As a result, the staff has confirmed that firms may present gross characteristics alone, without net characteristics, if they meet the following criteria:

  1. The characteristic must be clearly identified as calculated without the deduction of fees and expenses.
  2. The advertisement must also present the total portfolio’s gross and net performance in a manner consistent with SEC requirements.
  3. The total portfolio’s performance must be given at least equal prominence to, and facilitate comparison with, the gross characteristic.
  4. The total portfolio’s performance must be calculated over a period that includes the entire period of the characteristic being presented.

As with extracted performance, these conditions help ensure that the presentation is not misleading, reducing the risk of enforcement action.

Bottom Line: A Practical Path Forward

This updated SEC guidance provides much-needed flexibility for investment managers, allowing for the presentation of gross-of-fee returns in a compliant manner. Firms that clearly disclose their approach and follow the specified conditions can reduce compliance burdens while still meeting investor protection standards. While this does not eliminate all complexities of the Marketing Rule, it does offer a practical solution that allows for more straightforward and meaningful performance reporting.

For firms navigating these changes, ensuring clear disclosures and maintaining compliance with the general prohibitions of the rule remains critical. Those who align their advertising materials with these guidelines can now confidently use gross-of-fee performance in a way that is both transparent and in compliance with regulatory requirements.

Questions?

If you have questions about calculating or presenting investment performance in a manner that complies with regulatory requirements or industry best practices, we would love to talk to you. Please feel free to email us at hello@longspeakadvisory.com.