How to Survive a Verification Part 3: Verification Testing

This article is part three of a three-part series on how to survive a GIPS verification. If you haven’t had a chance to read parts one and two, we recommend reading those first. The first part covers tips and tricks for setting up your verification for success. The second part covers recommendations for kicking off the verification and provides context about the initial data requests made by the verifier.

In this article, we describe how to get through the actual verification testing, which tends to be the most time-consuming aspect of a GIPS verification. Specifically, we discuss how the testing sample is determined and then dive into the details of each major testing area. Plus, we include advice to help you determine what to provide to satisfy the verifier’s requests.

How the Testing Sample is Determined

The end goal of verification is the opinion letter that attests to “whether the firm’s policies and procedures related to composite and pooled fund maintenance, as well as the calculation, presentation, and distribution of performance, have been designed in compliance with the GIPS standards and have been implemented on a firm-wide basis.”

At this stage, the verifier has already reviewed your firm’s GIPS policies and procedures and likely confirmed that they have been adequately designed. The focus now is on whether these policies have “been implemented on a firm-wide basis.”

To test this, a sample must be selected. The size of the sample depends primarily on three things:

1. The number of portfolios managed
2. The number of composites maintained  
3. The verifier’s risk assessment of your firm

If you have had many errors in prior verifications or if your initial data provided to kick-off the verification had errors, the risk assessment will be high and the sample selection will likely be larger than average.

With that in mind, it is always a good idea to do your own review of your firm’s policies and procedures and data prior to providing anything to the verification firm. Even if issues were identified in the past, starting this pre-review now can help the current verification go faster and with less errors, potentially lowering the risk assessment and sample size for future verification periods.

What to expect with the Verification Testing Request

All verification firms are different in how they structure their testing process. They may have different names for the types of testing they do, but the main purpose is the same. The most typical types of testing include:

• Membership Testing
  • Entry Testing
  • Exit Testing
  • Outlier Testing
• Non-Discretionary Testing
• Return and Market Value Testing

Before diving into pulling and providing the requested documents, it is important to review the full request to ensure you are clear on what the verifier is trying to confirm. When not sure what to provide, it may be helpful to have a call with the verifier to discuss what is available. Knowing exactly what the verifier is trying to prove out makes it much easier to provide meaningful support. Blindly providing documents that may not tell the full story of what’s happening with the selected portfolio may lead to more questions/follow up.

Keep in mind that, the verifier prefers reviewing the most independent information available. For example, a contract signed by the client is typically preferred over an internal memo, but signed documentation from the client is not always available. To give you an idea of how to determine what to provide, below is a hierarchy of the preferred support:

1. Dated Correspondence Written or Signed by Client that Supports the Selected Testing Item
   • Contract and/or investment guidelines
   • Signed Investment Policy Statement
   • Termination letter
   • Email written by the client
2. Dated Notes Written by your Firm at the Time the Selected Testing Item Occurred
   • Email from your firm to the client
   • Email sent internally documenting the issue selected for testing
   • CRM notes written by your firm
   • Memo written by your firm saved to the client’s file
3. Written Explanation Created by Your Firm Now
   • A memo can be written now documenting support for the selected testing item. This is only acceptable if the person writing the memo has knowledge of the issue that can be documented to support the testing (e.g., a recollection of a verbal request from the client that was never documented) and if the other items above are not available. This should only be used in rare occasions as a last resort.

The following sections discuss each of the most common testing areas in more detail.

The purpose of membership testing is to confirm that portfolios are included in the correct composite for the correct time period, as determined by your firm’s GIPS policies and procedures. It is important to remember that any movement in and out of composites should always tie back to policies outlined in your firm’s GIPS policies and procedures and should not be made based on discretionary changes to a portfolio made by the portfolio manager. For the GIPS standards, the consistent application of policies is key!

Membership Testing

Entry Testing

The purpose of entry testing is to confirm that your firm has consistently followed your stated membership policies and procedures for including portfolios in composites. The verifier’s testing approach is adjusted to match your documented policies and procedures for portfolios entering a composite. Portfolio inclusion is typically triggered due to one of the following reasons:

1. New portfolio opened
2. Portfolio increased in size and now meets the minimum asset level of the composite
3. Material restriction was removed from the portfolio
4. Re-inclusion of a portfolio following a significant cash flow

When pulling supporting documents for the verifier, it is important to consider the reason the portfolio entered (or re-entered) the composite. When providing supporting documents, make sure the documents demonstrate the reason for inclusion and provide additional written commentary when necessary to help the verifier gain a full understanding of the situation. This will speed up the verification process as it will cut down on the need for follow-up questions.

In testing that your firm has consistently followed your stated membership policies and procedures for including portfolios in composites, the verifier is primarily focused on the timing and placement of portfolios entering a composite. Testing timing involves the verifier confirming your stated policy is being followed in terms of when the portfolio should enter the composite. Testing placement involves the verifier confirming that the portfolio is added to a composite that aligns with the portfolio’s investment objective.

To confirm the timing of a new portfolio’s inclusion, the verifier typically requests several documents, most commonly the account’s transaction summary and contract. The transaction summary provides information about when the new portfolio was funded, and when the portfolio manager started investing in discretionary assets. The contract is used to assess when the discretionary contract was signed by your client and ensure the timeline makes sense.

Placement can be a bit more complicated in terms of support. The verifier typically requests several standard documents to support the placement. This is not an all-inclusive list, but items requested to support placement can include contracts, investment policy statements, portfolio holdings, fee schedules, client correspondence, CRM system notes, and/or internal memos.

Exit Testing

Opposite of entry testing, exit testing is the verifier’s testing of portfolios that leave a composite. Before diving into the documents your verifier may request, it’s best to figure out why the selected portfolio is exiting the composite. This background knowledge will help you focus on the type of documentation to provide your verifier.

Because portfolios selected for exit testing are not joining a composite, there is no placement considerations in this testing like there is for entry testing. Therefore, exit testing focuses on the timing of a portfolio’s removal from the composite and ensuring the reason for removal is valid. Whether the account lost discretion, terminated, changed its mandate, or violated a composite-specific policy, the verifier will be looking to test the timing of the event.

The most common types of documentation that can support the removal include updated contracts, client correspondence, internal memos, and transaction summaries. The transaction summary will show the true timing of the change to the portfolio, but when discretion is lost or a mandate is changed the verifier will also be looking for support that this change was client driven, which cannot be supported by a transaction summary alone.

It is important to note that unless a composite is redefined or a composite policy is broken (such as the portfolio dropping below a minimum asset level) any movement of portfolios in or out of composites must be client-driven. That is, support for a portfolio exiting a composite should include documentation of a client requesting to terminate management, change the strategy, or add some kind of material restriction.

For example, a client request to hold high levels of cash because of declining market conditions may be a reason to remove the portfolio from the composite, if holding such cash moves the account to non-discretionary status (as outlined in your GIPS policies & procedures). Alternatively, if a portfolio manager used their discretion to deviate from the documented composite description (e.g., holding higher cash levels than described in its strategy due to adverse market conditions), this is not considered a valid reason to remove a portfolio from the composite.

Another form of membership testing is outlier testing. In this analysis, instead of evaluating the movement of portfolios in or out of a composite, the verifier assesses portfolios with performance that deviates from its peers. The purpose of this testing is to confirm that the portfolio is correctly included in the composite, despite the difference in performance.

Outlier Testing

Performance deviations can happen for several reasons and are not necessarily a problem. For example, the following are a handful of common reasons for performance deviations:

1. A portfolio may have a large cash flow causing a temporary deviation. This is especially true for composites that do not have a significant cash flow policy
2. A portfolio may contain different holdings than others in the composite, despite having the same objective
3. Sometimes smaller portfolios may be more concentrated than larger portfolios (most commonly seen when the composite does not have a minimum asset level)
4. A portfolio may be subject to client-mandated restrictions that the firm has deemed immaterial to the implementation of the strategy

The verifier will want to gain comfort that the investment mandate for the portfolio is in line with the composite strategy, any client-mandated restrictions are not material enough for the portfolio to be considered non-discretionary, and no composite policies have been broken relating to minimum asset levels, significant cash flows, etc. In these situations, an explanation should be provided to the verifier to help them understand why the portfolio belongs in the composite, despite having different performance for the month tested.

Please note that if a portfolio is performing differently than its peers because of a restriction, the verifier will want to confirm that the restriction is client-mandated and that it is not breaking any rules outlined in your firm’s definition of discretion within your GIPS policies & procedures. If the restriction is material and breaks your firm’s rules for discretion, then the portfolio should not be in the composite and will need to be removed.

Non-Discretionary Testing

While membership testing focuses on portfolios in (or moving in and out of) composites, non-discretionary testing focuses on confirming that portfolios not in composites have a valid reason to be excluded.

Unless a portfolio is deemed non-discretionary, all fee-paying segregated accounts must be included in a composite. Any account excluded from all composites should have a client-driven reason for the exclusion (e.g., a material restriction or a request for a custom mandate). As mentioned, deviations from the strategy made by the portfolio manager’s discretion are not valid reasons to exclude accounts from composites.

The verifier’s sample of non-discretionary portfolios will come directly from your AUM report or list of non-discretionary portfolios. The verifier is typically looking for non-discretionary portfolios they have never tested (many verification firms track portfolios they have tested in prior years), larger non-discretionary portfolios, and non-discretionary portfolios with unique/different reasons listed for being excluded. Because the list of non-discretionary portfolios is a snap shot in time, it will likely include many exclusion reasons– whether it is a long-term restriction, short-term restriction, or a violation of composite policies like a minimum asset level or a significant cash flow policy.

In addition to providing the reason the portfolio is non-discretionary, common requests from a verifier include contracts, investment policy statements and portfolio holdings reports. Verifiers use contracts and investment policy statements to find any documented restrictions in the paperwork. Often, this is clearly outlined in these onboarding documents, but this is not always the case. If restrictions are not clearly documented in these files, the verifier will likely request CRM notes, email correspondence with the client, and/or an internal memo to document the restriction in place. A portfolio-holdings report typically is used to see if any noted restriction is evident in the holdings and management of the account.

Portfolio-Level Return Testing

There are two main goals of portfolio-level return testing:

1. Confirmation that the input data used in the calculation can be independently supported
2. Verification that the calculation methodology outlined in the firm’s GIPS policies and procedures can be applied to the input data to achieve materially the same performance result as your firm

For a firm-wide verification, verifiers will likely have you provide a sample of custodial records in addition to portfolio accounting system reports to gain comfort that the input data used in the performance calculations can be independently supported. If you are having a performance examination on a composite in addition to the firm-wide verification, then this sample will likely be greatly increased.

The verifier uses the portfolio’s custodial statement in conjunction with the corresponding system holdings report and transaction summary to confirm whether market values and transaction activity, including trades, cash flows, fees and expenses, match between the two documents. If there are differences in the timing or amounts of transactions, the verifier will likely have follow-up questions to gain and understanding as to why such differences exist. Lastly, the custodial statement can also be used to confirm whether the portfolio is paying commissions on a per-trade basis.

The portfolio’s fee schedule may also be requested. If the composite calculates net-of-fee returns using actual investment management fees, the verifier will look at the portfolio’s gross and net-of-fee returns to ensure they are in line with the portfolio’s fee schedule. If the spread between gross- and net-of-fee returns does not reconcile with the fee schedule, there will be follow-up questions to figure out why the difference exists and if there are any other fees/expenses impacting the returns that need to be considered.

If your composite net-of-fee returns are calculated with model fees, the verifier will compare the provided fee schedule against the applied model fee. If the model fee is higher than the provided fee schedule, there will likely be no further questions. However, if the fee schedule is higher than the applied model fee, the verifier will likely need to do some alternative testing to ensure that the model fee applied is appropriate.

Once the input data is validated, the verifier will apply the calculation methodology outlined in your firm’s GIPS policies and procedures to confirm they can achieve materially the same performance results. Specifically, the verifier is looking at the treatment of external cash flows, fees, withholdings tax, and interest and dividend accruals to ensure the treatment of each meets the requirements of the GIPS standards and matches your firm’s policies and procedures.

If the verifier is unable to recalculate the returns following the methodology outlined in your GIPS policies and procedures and you believe the timing and amount of all transactions are consistent between your system and what the verifier is using, you should double check that your policies accurately describe your current system settings. For example, have you accurately documented whether external cash flows are accounted for as of the beginning of day or end of day, whether dividends are accounted for as of ex-date or payment date, whether performance is reduced by withholdings taxes, what size cash flows trigger revaluation, or any other settings that could trigger a difference in the performance calculation?

This is an important part of the verification testing as it confirms that an appropriate calculation methodology, acceptable under the requirements of the GIPS standards, has been consistently applied to the portfolios across your firm. Any material differences identified in this testing will need to be resolved before the verification can be completed.

Wrapping up the Verification

Once all testing procedures are complete, most verification firms will conduct their own internal quality control review to ensure the engagement team adequately addressed all testing items before officially signing-off. During this review some additional questions may arise to satisfy the reviewer, but the testing should be materially complete at this point. It is helpful if you can be prepared to answer questions and provide any last-minute document requests in a timely fashion to help move the project across the finish line.

Once all internal reviews have been completed and signed off, the verifier will send a representation letter to your firm. The representation letter is essentially a request for your firm’s attestation that, to the best of your knowledge, everything provided during the verification was accurate and complete. This is a necessary step that all verification firms require you to complete before the verification opinion letter can be issued.

After the representations letter has been attested to by your firm, the opinion letter should be issued shortly thereafter. This wraps up your verification project. Time to celebrate with your team and hopefully enjoy a bit of time away from the verification project before the next annual project begins.

One final recommendation is to have a debrief with your team and any consultants you work with to maintain GIPS compliance. Discuss what went well and what areas held up the verification project. If any areas held up the timeline, this is a good opportunity to consider ways to improve procedures and ongoing composite/data reviews. Planning ahead and implementing improved processes based on what was learned during the verification will help future verifications continue to get smoother over time.

Conclusion

The challenges faced when going through a GIPS verification can vary depending on your firm’s structure/size, the types of products you manage, and the verification firm used. This series was intended to provide a general overview of the most typical verification process and share tips and tricks for helping simplify your verification. If you have questions unique to your verification that we did not cover, please reach out to us to learn more. If you need assistance with a GIPS compliance, Longs Peak is here to help. We have helped hundreds of firms become GIPS complaint and maintain that compliance on an ongoing basis. We are happy to assist your firm with all of its needs relating to the calculation and presentation of investment performance.

You can email matt@longspeakadvisory.com or sean@longpseakadvisory.com with questions or check out our website for more information.