Start with the decisions the system must support
Agree the business problems and priority identity use cases before assembling a feature list. A contractor onboarding process, an access-review campaign and an administrator elevation request involve different data, owners and controls. For each, define the outcome and constraints. Separate mandatory requirements from preferences so that a high score elsewhere cannot hide a critical gap.
Give every supplier the same scenarios
Ask suppliers to walk through the same situations using comparable inputs. Include a joiner with incomplete data, a mover who no longer needs old permissions and a removal that fails in the target application. Ask who sees the failure, who resolves it and how completion is evidenced. Record which steps were demonstrated, explained or left for later validation.
Distinguish product capability from delivery effort
For every important requirement, ask whether it is available as standard, requires configuration, needs a separate component or depends on custom work. Request the assumptions and dependencies behind the answer. Clarify who will create, test and maintain the integration. Treat an assertion that something is possible as a question about scope, evidence and ownership, not as proof it is ready.
Score with evidence and a consistent method
Set criteria and weights before demonstrations and explain the scoring scale to evaluators. Maintain an evidence log beside the scores. A sample scale might distinguish no evidence, a partially met requirement and a demonstrated fit; adapt it to your procurement process. Discuss material differences between assessors and retain the reasons for the final score.
Evaluate the operating model
Ask how your team will manage failed jobs, access exceptions, releases, upgrades and changes to connected applications. Review training and handover proposals with the intended service owners. Include non-production environments and testing arrangements in the evaluation. A platform that fits a demonstration still needs a workable support model in your organisation.
Turn selection findings into delivery commitments
Before implementation, resolve open assumptions about scope, application volumes, identity populations and acceptance. Identify what the supplier provides and what your organisation must supply. Carry evaluation findings into requirements and delivery planning, with unresolved questions assigned to owners. Procurement and commercial teams should review the relevant commitments through their own approval process.
A useful evaluation record
Keep the requirements baseline, scoring method, demonstration evidence, open questions and decision rationale together. This record helps new delivery team members understand why a solution was selected and which conditions remain important. Revisit it when a proposed scope change affects an assumption that influenced the original decision.
This is a practical starting point for discussion, not a substitute for your organisation’s policies or product-specific implementation documentation.