Author: Savas Tutumlu, Co-Founder & CTO
Experience: MIT-trained • Led technical reviews for 100+ vendor evaluations • Built and audited large-scale systems
Published: November 17, 2025 • Reading time: 10 minutes
Every vendor looks great in a portfolio slide. The real question is: can their team design, build, and maintain the specific system you need—under real-world constraints.
This checklist is the one I wish every non-technical founder, COO, or investor used before signing a big software contract. It focuses on the areas that actually predict success: architecture, process, security, and ownership.
Quick Checklist: 10 Questions to Ask Every Vendor
- Who will be my lead architect and how many similar systems have they shipped?
- Can you show architecture diagrams for 1–2 comparable projects?
- What is your approach to testing (unit, integration, end-to-end)?
- How do you handle security reviews and vulnerability scanning?
- What happens if a senior engineer leaves mid-project?
- Who owns the source code, infrastructure accounts, and CI/CD pipelines?
- What is your incident response process if something breaks in production?
- How do you estimate work—and how often are you off by more than 20%?
- Can I speak to 2–3 technical stakeholders at current clients?
- Where do you document architecture and decisions (ADR, runbooks, etc.)?
1. Team Composition & Seniority
Sales will talk about “our team.” Your job is to uncover who actually touches your code and how senior they are.
- Ask for an org chart for your project: architect, tech lead, senior devs, juniors, QA, PM.
- Clarify which roles are full-time on your project vs shared.
- Ask how they handle turnover—who can step in without losing context?
Red flag: you never meet the person who will own architecture, only a salesperson and “project manager.”
2. Architecture & Technology Choices
Good vendors can explain architecture in plain language. Ask them to walk you through:
- How they’d decompose your system (modules, services, boundaries).
- Where they expect scale or complexity bottlenecks.
- How they plan for observability (logging, metrics, tracing).
If you’ve already prepared an RFP, reference it here. This article pairs well with our software development RFP template so vendors respond at the right level of detail.
3. Delivery Process & Quality Practices
Ask vendors to draw their delivery process—from backlog to production—and dig into:
- Branching strategy (GitFlow, trunk-based) and code review norms.
- Automated testing coverage and tools.
- Release cadence and rollback procedures.
- How they handle change requests and scope creep.
Then connect it back to risk: “Show me how this process makes a $150K fixed-bid project more predictable than hourly work.” Our upcoming article on fixed‑bid vs T&M vs dedicated teams goes deeper on this.
4. Security, Compliance, and Data Handling
Security posture is non-negotiable for anything touching customer data, payments, or regulated industries.
- Ask for a summary of their secure coding practices (OWASP awareness, code scanning tools).
- Clarify how credentials, API keys, and secrets are stored and rotated.
- If relevant, ask about HIPAA/SOC 2/GDPR experience and whether they’ve passed external audits.
Red flag: “We use HTTPS and a firewall; that’s usually enough.”
5. Code, Infrastructure & Knowledge Ownership
A common failure mode: vendor owns everything and you’re effectively locked in.
- Ensure repositories live under your GitHub/GitLab organization, with vendor access.
- Cloud resources should live in your AWS/Azure/GCP accounts.
- Require written handover materials: architecture overview, runbooks, onboarding guides.
At Stratagem Systems, we always structure engagements so you could continue without us. That confidence is a useful litmus test when comparing vendors.
6. Reference Checks That Go Beyond “They Were Great”
Ask to speak with at least one technical stakeholder at a current or recent client—CTO, tech lead, or senior engineer. Questions to ask:
- “What surprised you—good or bad—about working with this team?”
- “What would you do differently if you were starting the project again with them?”
- “How do they handle production incidents and tough conversations?”
7. Turn This Checklist Into a Scoring Rubric
To avoid purely gut-based decisions, build a simple scoring sheet with 4–6 categories:
- Architecture & technical fit.
- Team seniority & communication.
- Security & reliability posture.
- Process & project management.
- Cultural fit and transparency.
Score each vendor 1–5 per category, add short notes, and discuss as a team. This is far more robust than just “who is cheapest” or “who felt good on the call.”
8. Next Steps
Use this checklist as a companion to your RFP and pricing analysis:
- Draft or refine your RFP using our RFP template.
- Estimate budget ranges from our company pricing guide.
- Run technical due diligence with this checklist before you sign anything.
If you’d like a second opinion on a proposal or vendor, you can share it with us during a discovery call. We’re happy to give blunt feedback—even if the right answer is “this isn’t a fit for Stratagem, but here’s what to watch out for.”