Technology Scoring Tools

The below example tools and illustrations can provide an objective, structured framework for evaluating and prioritizing candidate projects and technologies in context of an organization's mission, vision, goals, objectives, management priorities, financial and staff resources, risk management criteria and organizational capabilities.

Tool Structure

Step 1: Screen Candidate Projects

Screen #1: Does this project or technology meet the primary goal?
Screen #2: Can this project or technology achieve the targeted benefits within the required timeframe?

Step 2: Develop Weighted Scoring Criteria

Identify Key Criteria that will determine the relative ranking of candidate projects or technologies and apply weights to the various criteria. See example criteria below.

Step 3: Score and Rank Candidate Projects

Organize candidate projects and technologies by "Type" and use the Weighted Scoring Criteria to score each candidate project or technology (In this example the projected timeframe to achieve benefits was used to organize porjects).

Step 4: Review Ranked List of Candidate Projects and Technologies

Review list in order of highest to lowest scores.

If ranked list appears reasonable, proceed to development of Implementation Plan.
If ranked list does not appear reasonable, adjust Weighted Scoring Criteria and re-score.

(Note: An iterative process is often needed to calibrate a new scoring tool.)

Step 5: Develop Implementation Plan

Develop plan for implementing projects and technologies that appear to provide high potential benefits within the organization's investment and risk management policies.

Customizing Weighted Criteria

The examples provided are illustrations only. The specific criteria used and their relative weights should be customized to each individual or organization’s specific goals and objectives for an activity of this kind.

The sample scoresheet provided on the next page seeks to estimate values for two factors:

Benefits attempt to evaluate the ability of any particular project or technology to support or advance California’s resource, economic and environmental priorities.
Adoption Challenges attempt to understand the types of challenges that targeted adopters will need to address in order to successfully implement the proposed project or technology.

Structuring the Score Sheet

There is no limit to the number of criteria that can be included, although the scoring process becomes much more difficult with the number of criteria.

Similarly, weights can be applied at multiple levels; but again, too much complexity may thwart one of the objectives for this type of process, which is to ensure transparency and objectivity.

Further, there may be more than one scoresheet – for example, different types of technologies with very different purposes may very well have different criteria (for example, technologies that address drought resilience vs. electric reliability).

Calibration

Especially if this is the first time that this score sheet is being used, calibration is usually needed.

After the first round of scoring (a sample can be used to calibrate the criteria and weights), the portfolio team reviews the relative ranking of various projects and technologies to determine whether the goals and objectives of the scoring process were met. If the purpose was to develop a portfolio of technologies that have high potential to advance drought resilience, a scoresheet that ranks AWG higher than F&B should be scrutinized and revised as needed to assure the appropriate result.

Remember this is just a Tool

The process of collectively identifying and ascribing weights to evaluation criteria helps to levelize participants’ knowledge and understanding of the portfolio’s goals and objectives, and the portfolio developer’s needs and interests. If the tool does not seem to be producing a rational outcome, change the criteria and weight and re-score some of the candidate technologies. This is usually an iterative (and sometimes frustrating) process; but with each iteration, new insights are gained as to the types of criteria that should be included and their relative importance to the evaluation process.

A sample scoresheet designed to rank candidate drought resilient technologies has been provided below, along with illustrative scoring of two technologies with very different characteristics:

Sample Scoresheet: Drought Resilience

Scoring Example 1: Atmospheric Water Geneartor

AWGs condense water from humidity in ambient air; combined with ultrafiltration (reverse osmosis), AWGs produce high quality emergency potable drinking water on-site. Download the scoring example below.

Download Here

Scoring Example 2: Food & Beverage Water Recycling/Reuse

Several technologies are available today that enable cost-effective on-site recycle/reuse of F&B process water. One technology uses proven physical-chemical processes to efficiently remove solids from F&B process water effluent. Download the scoring example below.

Download Here