Part III: We Compared AdviserGPT and the Leading LLMs for RFP Completion. For Enterprise-class Institutional Asset Managers.

Key Takeaways

• All three models maintained impressive semantic similarity at the Enterprise level (cosine score > 0.80), confirming that any of them can effectively capture or clone your firm's tone and voice regardless of document volume or strategy complexity.

• Factual accuracy decreased across all models as task complexity grew. While specialized tools maintained higher reliability than the standard frontier models, all systems showed increased risk profiles, underscoring the necessity of human oversight for enterprise-level RFP completion.

• Response times followed prior patterns. AdviserGPT answered the Golden RFP in bulk in approximately 3 minutes, while Claude took 20 minutes and Copilot took 1 hour after requiring multiple rounds of re-prompting. 

• The right tool for enterprise-class managers depends on your firm's technical resources and budget: a fully forward-engineered and maintained horizontal LLM deployment may be viable for firms with dedicated technical teams and expertise, while firms without that infrastructure will likely find a purpose-built tool reduces verification overhead. No tool fully eliminates the need for human review at this level of questionnaire volume and complexity.

In our prior comparison and blog post, we dove deeper into our benchmark comparison experiments comparing Anthropic Claude (Opus 4.8), Microsoft Copilot, and AdviserGPT for RFP/DDQ workflows in the medium-sized manager segment. We measured additional quantitative variables: Grounding Rate (percentage of statements with an exact match similarity score > 0.75) and Expanded Risk Rate (hallucinations, contradictions, unverified information) as well as qualitative aspects of each model including time to generate a complete response, context/token window constraints, and each model’s ability to handle large document uploads. 

Recall from our prior experiments that because the mid-sized manager segment contained over 10X more source documents and multiple investment strategies than the original emerging manager comparison, we provided index files to Claude and Copilot to assist with information retrieval and to minimize token usage. Our quantitative results for the medium segment echoed those of the emerging segment; all three models can successfully mimic your firm’s language and tone of voice, but the out-of-the-box frontier models struggled with accuracy. We found that AdviserGPT provided answers much quicker than the standard models, and while speed alone isn’t important, coupled with the other variables, our results show that specialized tools handle larger source document volumes more effectively. 

In this post, we update our series with the enterprise-class manager segment ($25B+ AUM), diving deeper into the quantitative statistics of semantic similarity and hallucination analysis across 43 documents, spanning nine US equity strategies, and 25 Golden RFP questions across many functional areas. Like the medium-segment test, Claude and Copilot were given index files to help with large-volume uploads and to optimize token efficiency. Additionally, to simulate the RFP completion process at the enterprise level, each model was required to filter its sources down to a specific US Equity strategy when answering certain investment process questions. 

In addition to our previous post, we continue our observations of user experience factors: time to produce a response, difficulty with large document volumes, hitting context window ceilings early, in order to provide a well-rounded assessment of each AI approach. 

Our goal to provide an objective comparison between AdviserGPT and out-of-the-box LLMs with light configuration did not change. Note that AdviserGPT utilizes leading frontier models such as Google Gemini, ChatGPT, and Claude as a part of its system.

Which AI Tool is the Best for Enterprise-class Asset Managers?

Analysis of Hallucinations and Semantic Consistency

In comparison to our smaller segment tests, the quantitative results for the enterprise segment support the same conclusion. On the surface, all three models perform well in capturing your firm’s tone and voice (cosine score > 0.80), with Copilot improving from the medium-size manager test while AdviserGPT and Claude remain comparable.

Assessing Precision and Factual Accuracy

Where the models diverge again is factual grounding, and that divergence is more pronounced with the enterprise-class test than in the medium or emerging segments. AdviserGPT achieved a Grounding Rate of 60.81%, a drop from 75.60% in the medium-size segment, indicating that even specialized tools show reduced precision as document volume and strategy complexity increase. Claude and Copilot followed at 15.73% and 10.31% respectively, down considerably from 40.12% and 32.38% in the medium-segment test. 

The supported statement counts tell a similar story. AdviserGPT produced 86 strongly supported statements out of 148 total sentences, while Claude produced 12 out of 178 and Copilot produced 8 out of 223. Despite a significant difference in supported statements, all three models also produced 32 or more weakly-supported statements, reinforcing that enterprise-class complexity increases verification requirements across the board, not just for horizontal models. Unlike AdviserGPT, which is built to retrieve only from source documents, Claude and Copilot rely on prompt instructions alone to stay grounded, creating instructions that break down as document volume grows, leading to more statements, more assumptions, and higher hallucination rates.

Evaluating Risk at the Enterprise Level

When unverified information is factored into the risk calculation along with hallucinations and contradictions, the resulting Expanded Risk Rate shows a widening gap between the models at the enterprise level. Claude's Expanded Risk Rate reaches 39.89% and Copilot's reaches 44.84%, compared to 17.57% for AdviserGPT. For context, AdviserGPT's Expanded Risk Rate more than tripled from the Medium-sized segment (5.36% to 17.57%), which reflects the added complexity and variability of enterprise-level source material.

As with previous tests, all three models produce output that benefits from human review. The practical difference at enterprise scale is how much review is required. A higher Expanded Risk Rate translates to more time spent verifying statements before submission, a cost that compounds quickly across the volume and complexity of RFPs and DDQs typical of an enterprise level firm. Off-the-shelf models configured with proper RAG infrastructure can reduce that overhead, but as noted previously, it requires meaningful technical and financial investment. AdviserGPT's pre-built retrieval and validation layer reduces the verification burden relative to a partially configured horizontal model, though the 17.57% Expanded Risk Rate makes clear it does not eliminate the need for review.

User Experience Considerations

For enterprise-class managers, the qualitative variables matter even more. While all models received identical prompts and source documents, response time varied significantly. AdviserGPT provided the response across multiple questions and strategies in less than 5 minutes for a 25 question RFP, similar to the results in the medium-size test. Claude followed with a response provided in 20 minutes, and Copilot took over an hour, requiring multiple rounds of re-prompting after reaching the maximum context window. While a medium-size firm might face some frustration and delays with context windows, for an enterprise firm, these issues are magnified. The added complexity of prompting based on document volume, number of questions, and specific strategies can require multiple rounds of re-runs not just one, which can cause your RFP tool to forget the most important parts of a response. In addition, the 25 question Golden RFP that we utilized is on the smaller side for enterprise-class mandates. RFPs can easily approach 100 questions and more, along with many data tables and data elements, which will significantly increase response times for both Claude and Copilot in particular.

What This Means for Enterprise-Class Managers

Across all three segments: emerging, medium-size, and enterprise, the pattern is consistent: semantic similarity is within a reasonable 10% range across models, while factual grounding, risk rates diverge meaningfully, with that divergence widening as document volume and strategy complexity increase.

For enterprise managers specifically, the right tool still depends on your firm's technical resources and operational constraints. If your firm has the infrastructure, budget, and timeline to build it, a fully RAG-engineered deployment of Claude or Copilot may perform better than what our tests reflect, and may be the first choice path for firms with dedicated technical teams and budget. However, the verification overhead associated with partially configured horizontal models becomes a significant operational consideration. A specialized tool like AdviserGPT reduces that burden from the start, though as this segment's results show, it does not eliminate the need for human review.

Our takeaway thus far from this series is that the performance gap between specialized and horizontal models grows as complexity increases, and that the cost of bridging that gap with infrastructure investment is a real, expensive variable every firm needs to weigh against its own resources. For our next comparison study, we will look into data-intensive DDQs instead of the mainly qualitative RFPs we have focused on thus far.