Problem-Solving Services - An Overview

          First in thread      Previous in thread

Introduction

Many problems that arise in industry and research are challenging to solve correctly, on schedule, and within budget. Reasons for these challenges include the complexity of the problem, the complexity of an algorithm, the availability of several algorithms for solving instances of a problem, and the rate of innovation and improvement in the development of algorithms.

Many problems are NP-complete and are thought to present any algorithm with instances that require exponential time to solve, even though most instances may be solved inexpensively. Other problems (e.g. graph isomorphism, factoring) have instances that are expensive to solve using currently available algorithms.

Before undertaking the solution of a problem instance, estimation of the resources that may be required and the risk of the solution effort failing may be desired. There are often several parameterized algorithms that may be used in attempts to solve a problem instance. Efficient analysis of the instance may permit the selection of those algorithms and parameters that are most likely to be effective in the solution effort.

Some algorithms are complex enough to be difficult to implement correctly and efficiently, and many algorithms are difficult to implement so that they can handle large instances of problems. Adapting algorithms for parallel or distributed processing, so that they can take advantage of frequent improvements in processing platforms, requires expertise in design, implementation, and testing.

Few development shops and few library providers are able to provide certifiable implementations of algorithms and few if any are able to provide certifiable implementations of state-of-the-art algorithms. Few are able to make use of algorithms that evaluate a problem instance and dynamically plan for its efficient solution, making use of the best tools available. Few make use of prior experience to inform and tune the development of solution plans.

I suggest that offering a problem-solving service is the best means of making problem-solving software for challenging problems available and propose that the design and implementation of such a service be undertaken by a company adequate to the task. I believe that the potential advantages of such a service to both the service provider and its clients are great and that it would be profitable and beneficial to all concerned if well executed.

The customer's point of view

At its simplest, a client sends to the service an instance of a problem, any parameters or constraints, and a description of the information that it wants returned. The service sends back a proposed budget for the solution effort and a final solution request / budget is negotiated. The service attempts the solution and returns the requested information, including solution(s) if found, to the client.

In more detail, here is how a successful solution effort might proceed as seen by a client:

• Client establishes an account with the service.
• Client brings up the service's catalog of problems, finds the problem that it wants the service to solve instances of, and learns how to exchange information with the service regarding instances of this problem.
• Client prepares solution request for a problem instance and sends it to the service.
• Client receives a response from the service containing price estimates, success probabilities, and proposed budgets for solution efforts (within a fixed price and beyond the fixed price), including the cases of the failure to find a solution and client-cancellation of the solution effort.
• Client modifies the solution request and sends it to service.
• Client receives a response containing new estimates and proposed budgets.
• Client sends approval of the proposed budgets to the service.
• Service initiates the solution effort for the problem instance.
• Client repeatedly queries the service for the solution effort status, receiving the status quickly.
• Client receives a solution response from the service, including a solution to the problem instance and metrics and other information that was asked for in the solution request.

A more complete version of this scenario will be provided later, including some high-level descriptions of what the service does.

Which problems are appropriate?

Not every problem is appropriate for sending to a problem-solving service for solution.

Problems that may not be appropriate for solution by a problem-solving service have one or more of the following characteristics:

• The problem does not have instances that are expensive to solve using an available algorithm
• At least one available algorithm is adequate for all instances that it may be applied to and straightforward to implement
• The problem needs to be solved within severe real-time constraints
• The problem needs to be solved on a platform that does not have reliable Internet access
• The problem needs to be solved as part of an application that will be sold or leased to parties that may not want to incur continuing problem-solving service costs

The challenges of solving instances of challenging problems

The central problem is this: Once we have more than one way of attacking a problem, how do we know which one to use?

Let's assume that we have two algorithms, A and B, for solving problem P. Neither is clearly more efficient than the other in general - each is expensive for some problem instances and inexpensive for others.

Our goal, then, is to minimize the cost of solving a sequence of problems on an engine.

Which algorithm, A or B, should we pick to solve instance I of problem P?

It really depends upon what we've learned about the behavior of A and B and what a characterization of I suggests in light of what we know about A and B.

We'll work with a problem-instance space - a space of one or more dimensions, each of which is a metric of a problem instance.

Each problem instance corresponds to a point in this space. We'll be looking into identifying regions of this space that are 'safe' and ones that are 'risky'. - regions in which all instances are easy for algorithm X and regions in which some instances that are difficult for X are found.

Consider boolean expressions and the problem of determining whether a boolean expression is satisfiable (has an interpretation under which it evalutes to True). There are many obvious metrics of boolean expressions (e.g. the number of literals, variables, negators in the Negative Normal Form), several that are less obvious (e.g. the number of clauses in the CNF and DNF equivalents of the expression) and some that I've made up (e.g. the degree of coincidence).

Which of these descriptive metrics are good at predicting a property that we're interested in?

An example from boolean expressions

This example is borrowed from my May 31, 2008 post:

Applet displaying a beautiful rotating plot of four metrics on 2000 Boolean expressions

I found that a metric that I called 'degree of coincidence', based upon an aspect of the structure of the expression, was a great predictor for whether expressions were tautologies or contradictions.

The Java applet (hopefully) running above displays four pieces of information about several hundred randomly generated boolean expressions.

The color of each dot is the expression's logical class: tautologies, contradictions, and expressions that are both satisfiable and falsifiable.

This is a static view of similar data:

In this view, the horizontal axis is the log₁₀ of the number of clauses in the Conjunctive Normal Form (CNF) equivalent of the expression and the vertical axis is the log₁₀ of the number of clauses in the Disjunctive Normal Form (DNF) equivalent of the expression.

The third spatial dimension, projecting out from the screen for the static view, is the Degree of Coincidence for the expression.

The important thing to note is how well the Degree of Coincidence separates tautologies from contradictions.

This instance space has four dimensions, one for each of the four boolean expression metrics. The problem in this case is to determine the logical class of an expression and three other metrics were chosen as being possible predictive metrics of logical class.

A set of metrics doesn't have to be a good predictor in all regions of an instance space to be useful. Other sets of metrics may be better predictors in other regions of the space. Even if some regions of the the space have no good predictors of a property, we'll benefit from the existence of the ones that do.

I have often fallen into the trap of searching the perfect, golden algorithm for a challenging problem. To be sure, that remains my ultimate goal, but I've begun to figure out that judging an algorithm on that basis is misguided and disheartening.

An algorithm is designed to solve a problem. The algorithm will usually work better for some instances of that problem than others. Some instances may be solvable more efficiently than others, while other instances may not be solvable at all by the algorithm.

Many problems that interest me are considered very challenging because they belong to a complexity class called 'NP-complete'. No one knows whether the NP-complete problems have polynomial or exponential time complexity, but we do know that if any of these problems is polynomial, then all of them are polynomial, and that if any of these problems is exponential, then all of them are exponential. Determining and proving the complexity of this class of problems is one of the great problems in mathematics and computer science.

We can construct a problem-instance space (of one or more dimensions) that contains all instances of the problem as points. For simplicity, we'll assume for the moment that this space consists of three dimensions, two of which are spatial. The third dimension is a color that reflects the time required to solve the instance. Let tough problem instances (can't be solved within a reasonable time) be colored red and the other instances be colored blue. Our goal is to come up with an algorithm for which the instance space is solid blue.

A problem-instance space for Algorithm A and Algorithm B might look like this:

	Algorithm A		Algorithm B		Best of A and B

while the problem-instance space to the right shows what happens when we use whichever algorithm is most efficient for each instance.

If we follow the simple rule that we should use Algorithm A for any instance in the top half of the space and Algorithm B for any instance in the bottom half of the space, we are guaranteed to use an efficient algorithm on each instance.

The world is seldom so kind or simple, but many algorithms can make performance guarantees for regions of a specified problem-instance space. By developing several very different algorithms for solving the same problem and determining the regions of the space for which they are effective, we may be able to greatly reduce, if not eliminate, the region in which red points can appear for an algorithm that makes uses the best algorithm for each instance.

In the quest for finding polynomial-time worst-case algorithms for some challenging problems, I have taken to referring to the goal of finding such a patchwork algorithm as the goal of finding 'polynomial coverage'. Similarly for 'tractable coverage'.

In this way, algorithms that fail to provide the desired coverage throughout a problem-instance space, but which are able to guarantee the desired coverage for some region of that space, may prove to be quite valuable as part of a patchwork algorithm.

An approach involving metrics and algorithms

We're faced with the problem of using a collection A of algorithms and a collection B of bound algorithms (algorithms in A bound to a sequence of non-instance parameters for each algorithm, if any) to efficiently and effectively solve instances of a problem P. P and B have the following characteristics:

• No algorithm in B can efficiently solve all instances of P.
• Many instances of P can be efficiently solved by some algorithm in B.
• Most instances of P can be efficiently solved by some application sequence of algorithms in B.
• Some instances of P may not be solved efficiently by any application sequence of algorithms in B.

We need to figure out how use information derived from an instance I of P to solve I as efficiently as we can and, if I is very likely to be intractable to algorithms in B, to determine this and report failure as quickly as possible.

We can usually derive many metrics (M_I) from an instance of a problem. We may derive additional metrics (M_I,b) when we apply an algorithm b in B to the instance, including the amount of work (W) that was required to find a solution, if one was found within a specified work limit.

Some metrics in M_I may be good predictors of some metrics in M_I,b and of W, in particular. They may not be uniformly good predictors, however - they may only be good predictors for some regions of a problem instance space.

We often have no idea, however, of which metrics may be good predictors of metrics of interest and the ranges of a problem-instance space for which they are useful predictors.

The approach that I propose is:

• Devise a large number of metrics of instances of P
• Perform Monte Carlo studies by generating a large number of random problem instances, measuring metrics for each instance, applying each algorithm with randomized parameters and recording appropriate metrics
• Analyze the database of metrics, searching for metrics, alone or in combination, that are good predictors of metrics that we are interested in
• Incorporate the knowledge from this analysis into a planning component that can be used to guide the solution of a problem instance
• Solve each real problem instance as guided by the planning component, recording relevant metrics and incorporating the resulting data into the statistical database that will inform the planning component
• Study selected real problem instances 'off-line', subjecting them to the same experiments and data capture that were used during the Monte Carlo studies, and incorporating the resulting data into the statistical database that will inform the planning component
• Some problem instances will present significant challenges to the problem-solving service. These instances should be deeply studied and the system modified with what has been learned.

Most of this problem-solving approach can be fully automated today, and the devising and gathering of new metrics should be automatable in the future.

Monte Carlo studies with comprehensive algorithm sampling is time consuming and expensive, but it needs to be done once well and can be augmented with data from the processing of real problem instances and their off-line exploration in the future.

Once we have a sufficient understanding of our metric database, we can devise branching plans of attack for problem instances based upon their characteristics. We'll want to use inexpensive metrics on our instance that will begin to move us down the branches of this plan tree, reserving the more expensive metrics for when they are needed, if at all.

Next, we take a problem instance that we've been asked to solve, we can begin to follow the plan tree, making measurements of metrics and branching as guided by the plan and its underlying statistical database.

Suppose that algorithms C and D each take one real parameter in the range [0.0 .. 1.0]. Assuming that parallel processing is not available to us, we might, guided by our statistical database, execute a sequence of actions as follows for instance I:

1. Calculate Metric₁₄ and Metric₂₇ for I
2. Based on these values, calculate Metric₃ for I
3. Based on this value, apply Algorithm C with parameter 0.37 to I for up to 3.4 seconds, retaining its state if unsolved
4. Unsolved, so obtain Metric₈ and Metric₄₂ from the retained state
5. Based on these values, calculate Metric₅ for I
6. Based on this value, apply Algorithm A with parameter 0.71 until completion

The Adaptive Market pattern

I've frequently encountered the need to manage the use of available tools to perform a task effectively. I've generalized this into the Adaptive Market design pattern.

This pattern regards a community of agents as a free market in which those offering services and those desiring services are free to enter into contracts for the provision of services and for compensation for having done so. The market is adaptive in that many properties of the participants will change over time, including capabilities, capacities, resources, standards, reputation, metrics, tactics, and strategies. Adaptive markets may be nested as deeply as necessary.

The knowledge that the many participants in an adaptive market possess is localized and may be largely derived from experience. Implementations of adaptive markets for use in a problem-solving service are likely to be based upon a dynamic statistical database that may be initialized and periodically enriched by significant Monte Carlo studies.

A typical sequence of interactions with a market includes putting a task up for bid, evaluating submissions, selecting a contractor, negotiating a contract, executing the contract, and evaluating performance.

Problem-solving engines

A problem-solving engine is a software component that:

• Implements an algorithm to solve instances of a problem P
• Can provide an estimate/risk profile of the amount of work that it may be need to solve an instance of P
• Can attempt to solve an instance of P
• Can make effective use of a mechanism (e.g. blackboard) for tracking metrics and other information about an instance of P
• Can learn from its experience in trying to solve instances of P
• Can suspend and resume solution efforts on an instance of P

Knowledge about an instance and solution efforts for that instance can be obtained from a blackboard, which should be updated by the engine at the end of its work on the instance or upon suspension. Information from a blackboard may obviate some of the work that the engine would normally perform.

Knowledge from the blackboard may be used by adaptive markets to update the statistical and other models maintained at various levels of the system, including the problem-solving engines.

More than one occurrence of a problem-solving engine may be running at a time, each of which is tasked with work on a different instance of the problem.

Transforming instances between problems

All problems in the NP-Complete complexity class have the following characteristics:

• Each problem has many instances that are inexpensive to solve for some algorithms
• Each problem has some instances that are very expensive to solve for any existing algorithm
• All instances of a problem can be transformed into instances of any other NP-complete problem within polynomial time

It isn't that we transform one problem into another but that we transform an instance of one problem into an instance of another problem, solve that other-problem instance, and transform the solution back into the original problem context.

For example, suppose that we have a boolean expression whose satisfiability we want to determine but our program for determining boolean satisfiability is weak or missing. We do, however, have a very good program for solving instances of the k-Clique Exists problem. We can transform the boolean satisfiability problem instance into a k-clique exists problem instance, solve it, and port the solution back into the boolean satisfiabity realm.

There are a great many NP-complete problems, including some of my favorites (e.g. Boolean Satisfiability, k-Clique Exists), as well as classics such as the Traveling Salesman problem, Bin Packing, Multiprocessor Scheduling, Timetable Design, and Clustering.

Transformation of instances between problems isn't restricted to the NP-complete problems, but it is especially useful for them, for polynomial-time transformability is guaranteed.

An instance-transformation engine is a software component that:

• Implements an algorithm to transform instances of problem P₁ into instances of problem P₂
• Can provide an estimate/risk profile of the amount of work that it may be need to transform the instance
• Can transform an instance of problem P₁ into instances of problem P₂

An instance-transformation engine may, but is not required to:

• Make effective use of a mechanism (e.g. blackboard) for tracking metrics and other information about an instance of P
• Learn from its experience in trying to solve instances of P

Problem graphs

A problem graph consists of nodes, each of which represents a different problem, and directed edges between nodes, each of which represents a different transformation of instances of one problem to another.

Associated with each node is an adaptive market for solvers of the node's problem, either directly through solution engines at this node or indirectly through transformers and solution engines at other nodes.

Associated with each directed edge is an adaptive market for solvers of the source node's problem that includes the cost of transformation.

Modification of this graph is straightforward. Due to the localization of knowledge in the graph, nodes, edges, solution engines, and transformation engines may easily be added, removed, or modified.

A problem-solving system

I'd like to have a problem-solving system that can solve any of the NP-complete problems, as well as many other challenging problems.

This system would offer a catalog of problems that it can solve, although it may not have a good engine for many problems. The heart of the system is a problem graph that contains a node for each problem in the catalog.

This system would accept a problem instance to be solved, put the solution effort up for bid in the problem's node's adaptive market, choose the best bidder, negotiate a budget with the client, and, if approved, award the effort to the best bidder, returning the results and any solutions to the client.

Using a problem-solving service - an example

• Client establishes an account with the service.
• Client brings up the service's catalog of problems, finds the problem that it wants the service to solve instances of, and learns how to exchange information with the service regarding instances of this problem.
• Client prepares solution request for a problem instance and sends it to the service.
• Service prepares a solution request and puts it up for bid in the problem's adaptive market
• Agents in the problem's adaptive market, which may include proxies acting on behalf of other engines, evaluate the request and the accompanying instance, preparing bids if desired.
• Service evaluates the bids, and negotiates with the top few bidders as needed.
• Service prepares a response for the client.
• Client receives a response from the service containing price estimates, success probabilities, and proposed budgets for solution efforts (within a fixed price and beyond the fixed price), including the cases of the failure to find a solution and client-cancellation of the solution effort.
• Client modifies the solution request and sends it to service.
• Service prepares a modified solution request, puts it up for bid in the problem's adaptive market, evaluates the bids, negotiates with the top bidders if necessary, and prepares a response for the client.
• Client receives a response containing new estimates and proposed budgets.
• Client sends approval of the proposed budgets to the service.
• Service initiates the solution effort for the problem instance with the selected bidder.
• Selected bidder attempts solution of the problem instance
• Client repeatedly queries the service for the solution effort status, receiving the status quickly.
• Selected bidder ends the solution effort either by solving the instance or determining that it is intractable.
• Service provides feedback on the selected bidder.
• Client receives a solution response from the service, including a solution to the problem instance and metrics and other information that was asked for in the solution request.

Some challenging questions

Imagine a company that develops and sells access to software that solves challenging problems for customers. It offers a catalog of problems that its software can solve. What are some of the questions that need to be answered satisfactorily before a potential customer decides to use this software?

• Why should we pay someone else to solve our problems?
• Why should we continue to pay someone else to solve our problems on an onging basis?
• How do we know that your software will solve our problems correctly?
• How do we know that your software will be available when we need it?
• How difficult will it be to integrate your software with our software?
• Some of our problems could take extraordinary resources and time to solve.
  • How can we estimate the time and resources that may be needed?
  • How can we avoid financial exposure to solution efforts of expensive problem instances?
  • How can we remain informed of the status of the solution of a problem instance?
• How can we keep your software up to date?
• Many of our problem instances are related or similar - how can your software take advantage of this?

Some objectives of commercialization

There are many objectives that must be considered when designing a company that would provide problem-solving services. I'll focus on a few objectives that I consider fundamental, that address some of the questions that I have raised, and that support a decision to offer software as a service:

• Protect intellectual property
• Solve problems correctly and efficiently
• Accumulate and make effective use of knowledge that provides a competitive advantage
• Earn customers' trust
• Earn a significant profit

Protect intellectual property I know of no way of preserving the intellectual property in software that is accessible outside of a secure location. Anything can be reverse engineered, even software that is available only as a service. Not having access to the software's executables, however, makes reverse engineering significantly more difficult. Many of the algorithms that the service would use would be in the public domain, though several would undoubtedly be proprietary and be part of the company's intellectual property. The most important knowledge that the company will acquire over time will consist of effective predictive metrics and the statistical database derived from attempting to solve problem instances. Software as a service affords the best protection to intellectual property contained in software that is used by others. Solve problems correctly and efficiently Our up-to-date problem-solving software is the fastest path from problem to solution. Not every problem instance is solvable within a reasonable budget, but we will alert you to solution risks and manage them appropriately. Once an account has been established, the negotiation of problem solution efforts is quick and automatic. Given our large pool of resources, we can provide the level of service that you need. Solution verification is normally performed automatically as part of our service. In the event of an error or defect, our knowledge of the software version, platforms, request, and solution-effort history will allow us to identify and correct the defect as quickly as possible. Accumulate and make effective use of knowledge that provides a competitive advantage All problem solution efforts occur on our servers - we can retain solution-effort histories, metrics, and, with client permission, recalcitrant problem instances. Our algorithms will be informed by a statistical database of metrics from prior solution efforts and Monte Carlo studies. Experience-guided estimated and solution will result in increasing quality and a distinctive competitive advantage. Earn customers' trust Customers won't become customers and make use of our service until we earn their trust. They need to believe that we know our business and can provide the level and quality of service that we advertise. Our service needs to be up, robust, and reliable. Potential clients can experiement with our solution services. We publish the results of automatic validation of solution efforts on publicly available collections of problem instances. We also make available to clients the results of automatic validation of solution efforts on their collections of problem instances. Commissioning someone to attempt to solve an instance of an intractable problem is risky and exposes one financially. Our estimation and budgeting mechanism is designed to inform and to allow our clients to manage these risks effectively. Our estimation abilities are sharpened and improved with every solution effort our service performs. Our skill in estimation is key to attracting clients, keeping clients, and making a profit. We will keep clients only if we provide great customer service and extraordinary responsiveness. We should give serious consideration to client requests for additions or enhancements to the problem catalog or to our algorithm repetoire. Earn a significant profit In addition to providing an excellent and much-needed service, our goal is to earn a significant profit. Nearly all aspects of the service's operation are automated, including interaction with clients, account registration, solution-effort negotiation, solution efforts, and reporting. Experience-based and experience-guided solution efforts are key to our profitability. The accumulation, analysis, and organization of knowledge derived from experience will result in a major component of our intellectual property. In addition to its use in estimation and planning in production, this statistical knowledge can provide information on hot spots that could benefit from further research and development.

Some answered questions

Here are some answers to questions that potential customers should ask of a provider of problem-solving services:

Why should we pay someone else to solve our problems? Many of the algorithms we use are challenging to implement correctly and efficiently. Some of our algorithms are proprietary. The choice and effective use of these algorithms is informed by experience that we have in abundance. We can solve your problems today - there is no need to wait while you research, implement, and validate algorithms and their implementations. Why should we continue to pay someone else to solve our problems on an ongoing basis? Our algorithms and our implementation of them are used by many customers. They are regularly monitored and improved - corrections and improvements are available to all as quickly as possible. The expertise that we gain by solving a large number of problems for many clients is quickly available to solve your problems. How do we know that your software will solve our problems correctly? We validate our software on publicly accessible collections of problem instances. We will also validate our software as requested on problem instances that you provide. You may experiment with our software on a limited basis and test its correctness yourself. How do we know that your software will be available when we need it? Our software runs on professionally maintained clusters of servers that are robust, fail-resistant, and scalable. How difficult will it be to integrate our software with your software? All of our services are accessible via standard internet protocols and XML. Our site provides detailed examples of how to interact with our services to request problem solutions, negotiate, and receive responses. Customer support is always available How can we estimate the time and resources that might be needed? When you send us a problem to solve, we will automatically evaluate it and respond quickly with a free estimate of the resources that will be needed to solve it and a price schedule. Our estimates are based upon our exerience solving similar problems. If you need a solution unusually quickly or wish to have sufficient resources reserved for the solution of problems in the future, this can be negotiated. Many of the problems that we solve have instances that we will not be able to solve in budget that you require. Our price schedule will contain three schedules - one for solving the problem within budget, one for failing to solve the problem after exhausting the budget, and one for your cancellation of our solution effort once it has begun. Our effort to solve your problem will be measured in resource units. Our price schedule will specify a set rate for solving your problem within a specified number of resource units and a decreasing rate for additional resource units that are needed to attempt to solve your problem. If we are unable to solve your problem within the budget that you have approved, work on your problem may be preserved pending your decision to extend the budget, possibly with modified solution parameters. You may cancel the solution process at any time. How can we avoid financial exposure to solution efforts of expensive problem instances? Our continuing experience solving problem instances similar to the problem instances that you need to solve allows us to offer fair estimates that will improve with time. Our profits depend upon the accuracy of our ability to estimate the resources needed to solve a problem instance. The price schedule that we quote you will be based upon the specific problem instance that you need to have solved. The fixed rate for solving the problem within a specified number of resource units rewards us for ingenuity and knowledge while keeping us quite competitive. The cost of continuing past the fixed-rate threshold decreases with the amount of additional resources - we will absorb an increasing amount of that cost and use what we learn to improve our estimation capabilities. How can we remain informed of the status of the solution effort of a problem instance? You may query our service frequently in order to determine the status of the solution process for any of your active problems. You may use this status to update progress displays, decide whether to extend the budget, or decide to cancel the solution effort. How can we keep your software up to date? We keep our software up to date. When you ask to have a problem solved, we will use our most current tested and validated software to solve your problem. You can obtain information about the issues and validation history for the software that we use to solve any problem in our catalog from our site. How can your software take advantage of similarity in problem instances? Our software classifies problem instances using a dynamic statistical database and can use solution strategies and tactics that have been successful with similar problem instances. You may also specify a problem-instance category of your own choosing when you submit problems for solution - we will use this to further specialize the statistical data that we use to guide the solution of your problems.

Conclusion

As software-as-a-service becomes a viable means of providing access to software, providing access to algorithms and problem-solving services through the internet emerges as a promising business opportunity.

Many commonly occurring problems present enough challenging instances and financial risk to benefit from the expertise, resources, and innovation available from a premier problem-solving service provider.

The approach to a problem-solving service that was sketched in this document, including the Adaptive Market pattern and a problem graph that hosts problem-solving engines and problem-instance-transforming engines, offers a coherent and comprehensive framework could be the basis of a commercially successful enterprise.

The path to launching a problem-solving service includes planning, acquiring competence, achieving performance and availability, testing sufficiently to inspire confidence, gaining trust, learning from experience, and ongoing research and innovation.

          First in thread      Previous in thread

Copyright and Legal Notices

All Rights Reserved.