US20060111959A1 - Surveying apparatus and method for compensation reports - Google Patents

Surveying apparatus and method for compensation reports Download PDF

Info

Publication number
US20060111959A1
US20060111959A1 US10/541,156 US54115605A US2006111959A1 US 20060111959 A1 US20060111959 A1 US 20060111959A1 US 54115605 A US54115605 A US 54115605A US 2006111959 A1 US2006111959 A1 US 2006111959A1
Authority
US
United States
Prior art keywords
user
user profile
affinity
match
question
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US10/541,156
Inventor
Douglas Tarr
Joseph Giordano
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Payscale Inc
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US10/541,156 priority Critical patent/US20060111959A1/en
Assigned to PAYSCALE, INC. reassignment PAYSCALE, INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: GIORDANO, III, JOSEPH, TARR, DOUGLAS
Publication of US20060111959A1 publication Critical patent/US20060111959A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/10Office automation; Time management
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00Administration; Management
    • G06Q10/06Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
    • G06Q10/063Operations research, analysis or management
    • G06Q10/0639Performance analysis of employees; Performance analysis of enterprise or organisation operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201Market modelling; Market analysis; Collecting market data
    • G06Q30/0203Market surveys; Market polls
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/02Marketing; Price estimation or determination; Fundraising
    • G06Q30/0201Market modelling; Market analysis; Collecting market data
    • G06Q30/0204Market segmentation
    • G06Q30/0205Location or geographical consideration

Definitions

  • the following disclosure relates generally to correlating statistical records and more particularly to correlating compensation records to unique individual profiles.
  • an automated compensation service may categorize and calculate data showing that the average yearly salary of a “Computer Programmer Level 3” in Washington state is $64,250. This may or may not be applicable to a “Senior Application Software Engineer” with ten years of experience and special training in the skill C++, but because the closest answer describing the Senior Application Software Engineer's position in the initial survey was a “Computer Programmer Level 3,” the Senior Application Software Engineer has thus been categorized ineffectively, which removes any unique abilities he may possess.
  • the compensation analyst might have created a report for an isolated group called Computer Programmer Level 3.
  • the reports generated from such a compensation analysis are reasonably accurate.
  • the reports are essentially useless.
  • the data are by definition misapplied because any differences in the compared data are arbitrarily reflected in the compensation report. This introduces doubt on the user's part as to how closely he can trust the report's applicability.
  • Current compensation analysis techniques do not provide users with affordable, accurate, and personalized compensation reports. Job specific variables, critical to the accurate assessment of an individual's worth, are not correctly identified or uniformly applied.
  • profiles are used to produce individualized compensation reports.
  • a survey engine is used to produce profiles of individuals that identify the individuals' unique characteristics.
  • the survey engine incorporates a collaborative filtering engine that determines appropriate questions to ask the user during the survey, and also provides suggested possible answers.
  • the system allows for the use of open-text questions. Open-text questions allow for new answers to be input by the user, without the prior need for an administrator to pre-define the possible values for the system, as is typical in prior art.
  • the system incorporates affinity groups around profile attributes (question answers), providing a basis for gauging similarity of profiles for various comparison and aggregation purposes described herein.
  • a collaborative filtering engine is incorporated, which is both periodically modified by an administrator, and also tuned by users themselves based on their actions and responses.
  • New affinity groups associations of profiles
  • affinity groups are incorporated by the survey engine to suggest new questions and possible answers in a survey. Additionally, some affinity groups are generated automatically by the system, and finally by users themselves to create new interesting relation'ships among profiles.
  • the collaborative filtering engine enables the capture of profile attributes that are targeted compensation variables concerning a profile.
  • the system automatically incorporates new profiles into existing affinity groups, which allows more targeted survey questions and possible answers to be determined without requiring constant human training or intervention.
  • the system is unable to administer surveys accurately for users who do not fit into pre-defined categories, it is impossible to categorize every occupational variation because the system allows for open-text answers to questions, and because the system provides for different questions to be determined automatically and asked for differing types of user job profile, the collaborative filtering system, along with affinity groups and other requirements described herein, is employed. Because the collaborative filtering system allows for the system to make educated guesses within defined constraints, the system can handle new categorizations more effectively than a system wholly defined by a human administrator. Additionally, in this system, the administrator defines constraints that prohibit the survey from asking obviously wrong or out of place questions. An example of a constraint is requiring that if the user does not answer any of the suggested questions, a default question is always asked.
  • a new and surprising effect of using a collaborative filtering system to define a survey is that the system is accommodates a much larger population of data using a more targeted survey.
  • Previous implementations of compensation surveys relied on a smaller sample size, a broad survey with generalized questions, or a larger base of analysts to design and conduct surveys and categorize the data.
  • the survey was constrained purely by the human resources needed to conduct it.
  • the invention described herein is not constrained as such and requires far fewer human resources to conduct such a detailed survey across many different job categories.
  • Another key aspect of the invention includes the ability to search through the data that have been collected by the survey engine, using detailed search criteria defined by a search definition document.
  • the document applies a scoring and filtering mechanism that returns the most appropriate set of profiles based on the user's goal for the analysis.
  • the search document is useful because it allows an administrator to define natural relationships between profile attributes, such as those that apply routinely to the realm of compensation analysis of various classes of profiles.
  • the document is then interpreted by a software algorithm, and used during the retrieval of relevant profiles, which can then be used for tallying and reporting.
  • Multiple search definition documents can be created quickly, each one for a different goal. For example, one search goal may weight skills and certifications very highly. The results are useful for analyzing how to compare people with similar technical skills.
  • Another search definition document weights experience and education higher, which is useful in seeing how a user compares to those profiles who have similar experience and education levels.
  • a third example holds location constant, and compares a user to all other profiles with matching attributes in the same location.
  • a chart is defined as a series of values, such as skills, paired with a series of measures, such as average salary, median salary, standard deviation, etc.
  • a sample chart is called “Average Salary By Skill” and it lists each Skill along with an associated average salary.
  • a report is defined as a series of charts combined to provide an overall picture and analysis for a user. For instance, consider a report that aims to discover how a user compares with regard to skills and experience in similar jobs. This report incorporates many charts, as defined previously, which are combined into a format that gives a user a good analysis and understanding of the results of the user's search goal.
  • Prior art approaches are unable to automate the selection of charts within a compensation report, such as determining if “Average Salary by Practice Area” is applicable to a report presented for a particular user profile (it may only be applicable to a Lawyer, for instance).
  • the prior art requires that the administrator know in advance, every chart that is relevant for a particular user.
  • the invention described herein determines if a chart is relevant to a user based on the user's own profile, and displays it, and also only shows it if enough data exists for it to be valid statistically.
  • each of these compensation reports is compiled by a human analyst, created at expense, or details are ignored, leaving only the most common charts in the compensation report, such as “Average Salary by Occupation and Location,” which are often the least useful to an individual trying to compare his compensation against his peers.
  • the invention herein allows for a more targeted and relevant compensation report by focusing in an automated, scalable fashion, on attributes that are unique to a user, in addition to the attributes that are most common to all users.
  • FIG. 1 is an overview of a website implementation of the system according to the invention
  • FIG. 2 is a general user flow through the system according to the invention.
  • FIG. 3 is a specific user flow for the first time a user accesses the system according to the invention.
  • FIG. 4 is a flow for the system to suggest a FieldGroup to a user according to the invention.
  • FIG. 5 is a flow for saving a user's answer to a database according to the invention.
  • FIG. 6 is an exemplary profile according to the invention.
  • FIG. 7 is an example of the system suggesting a FieldGroup in the survey
  • FIG. 8 is a continuation of FIG. 7
  • FIG. 9 is an example of the system suggesting popular answers for a FieldGroup, in the survey according to the invention.
  • FIG. 10 is a continuation of FIG. 9 according to the invention.
  • FIG. 11 is a flow for calculating and summarizing data according to the invention.
  • FIG. 12 is a flow for storing tokenized data in a database according to the invention.
  • FIG. 13 is a flow for populating an affinity group according to the invention.
  • FIG. 14 is a flow for generating a custom report for a user according to the invention.
  • FIG. 15 is an overview of the different aspects of a profile search according to the invention.
  • FIG. 16 is a flow for a profile search according to the invention.
  • FIG. 17 is an example of an affinity group according to the invention.
  • FIG. 18 is a flow for the rules engine according to the invention.
  • FIG. 19 is a series of rules used in the rules engine according to the invention.
  • FIG. 20 is an exemplary apparatus in accordance with the disclosed invention being further connected to a network according to the invention.
  • FIG. 21 is an exemplary flowchart for the purpose of creating a user specific affinity group according to the invention.
  • FIG. 22 is an exemplary flowchart showing the profile search process according to the invention.
  • FIG. 23 is an exemplary XML code for profile matching according to the invention.
  • FIG. 24 is an exemplary flowchart the matching of a user profile to an affinity group according to the invention.
  • FIG. 25 is an exemplary flowchart for reporting of results of a matching process.
  • a method and system for providing targeted online compensation reports that accounts for unique individual job characteristics by using dynamic profiles is described in detail herein.
  • numerous specific details are provided for survey flow, affinities, levels, suggest FieldGroup, suggest popular answers, save answers, profile search, scoring system, report aggregation, and rules engine.
  • One skilled in the relevant art will recognize that the invention can be practiced without one or more of the specific details, or with other symbols, methods, etc.
  • well-known methods or techniques are not shown:, or are not described in detail, to avoid obscuring aspects of the invention.
  • Field A single piece of information, corresponding to a particular question asked by the system. Examples, “City,” “State,” “Bonus Amount,” “Skill,” and “Job Title.”
  • FieldGroup A set of related Fields that form a logical grouping of information into a single record. Also often referred to as a Question. Examples: “Salary” is a FieldGroup consisting of “Salary Amount,” “Currency,” and “Average Workweek.” “Job Location” is a FieldGroup consisting of “City,” “State,” and “Country.”
  • AnswerValue also Value, Answer or Attribute
  • AnswerValue A value (piece of information) corresponding to a Field, used as a response.
  • Profile A set of FieldGroups, Fields and AnswerValues that form a logical representation of an individual person or group of people.
  • Affinity Group (or Affinity)—A grouping of profiles that share common profile attributes, and further as shown in FIG. 17 . Example: “All People who work in Law or Legal Professions.”
  • Affinity Definition A Boolean representation of the common attributes shared by an affinity group.
  • Profile Search A detailed set of criteria used for matching profiles to other profiles and creating a scoring system that ranks the validity of the match.
  • Profile Search Document A document that encompasses the criteria defined for the profile search.
  • Report A set of charts combined in a specific layout to provide a detailed analysis of a profile search and comparison goal.
  • Every individual possesses unique distinguishing characteristics in their employment profile. These unique characteristics, even when seemingly minor, can correspond to differences in employment compensation. Being able to have a custom report that compares relevant characteristics for each individual, and having an understanding of what the market is willing to pay for the individual's abilities based on such comparison, is an important step in finding and effectively negotiating employment opportunities, as well as making informed career decisions.
  • the method and apparatus presented herein provide comparative compensation reports based on characteristics determined through a survey and a scored-attribute-matching search and reporting process.
  • FIG. 2 where a general user flow through the system is shown.
  • a user begins the investigation of their worth by accessing an Internet Website through a user interface, such as a personal computer, personal data assistant, or similar device. Once access has been established, the user conveys to the system his desired compensation comparison objectives/goals, as shown in FIG. 1 .
  • a survey engine also referred to as survey wizard, begins identifying the unique characteristics of the user that are most applicable to determining a compensation level.
  • the survey engine uses a combination of open and closed FieldGroups to create an individual profile.
  • An open FieldGroup is defined as having at least one Field for which the user may enter an answer in free text.
  • a closed FieldGroup is defined as having a Fields for which the user may only select from a list of predefined choices.
  • the affinity groups are discrete groupings of individual profiles based on Boolean criteria.
  • the Boolean criteria either simple or complex, are based on a combination of values from one or more attributes on a profile, such as job, skill, location, etc.
  • a user's profile attributes are compared against each affinity definition, and if the profile attribute meets the affinity definition criteria, the user is considered to be a member of that affinity group.
  • the survey engine or system, discussed in more detail below, through iteration, asks FieldGroups of the user until the engine determines no more FieldGroups should be asked.
  • the user's objectives/goals are confirmed through an initial set of questions, as shown in FIG. 1 .
  • the goal establishes broad areas that should be investigated by the survey engine.
  • the engine using this goal, suggests the next FieldGroup (Question) to be posed to the user, from the set of all available FieldGroups in the system shown in FIG. 4 .
  • the FieldGroup along, with possible popular answers obtained from previous questioning of different users, is presented to the user.
  • the system examines the answer offered by the user and, if appropriate, saves it in a database, as shown in FIG. 5 .
  • An affinity group is defined as a group of profiles defined by a set of profile criteria, called an affinity definition.
  • Affinity definitions are a combination of values from fields defined in the system also shown in FIG. 6 .
  • the corresponding set of profiles that match this definition is the affinity group.
  • the affinity definition is stored in a relational format, which is easily and quickly retrievable, and searchable.
  • One skilled in the art would recognize that such a definition could be stored in a wide variety of formats suitable to the task at hand, or modified for improved CPU performance.
  • a program quickly compares the user's profile to all affinity definitions stored in the system, and determines to which affinity groups a user belongs.
  • affinities can be created-which group numeric data by ranges. For instance, all people in a certain age range could be grouped together. Affinities are associations of data, defined by a human, an administrator, or users, which allow the system to create a more intelligent output for use by many of the sub-systems described herein. Additional detail is provided below in the form of flowcharts.
  • the survey engine is required to ask FieldGroups that are relevant for a particular user.
  • Many different job profiles have different attributes that affect one's compensation. For example, a CEO may need an answer to the FieldGroup (question) Company Revenue in his profile, while a Lawyer might need Practice Area or Bar Association Memberships.
  • the suggest FieldGroup algorithm is designed to ask pertinent questions of the user depending on the user's compensation analysis goal and the type of profile the user has, learned through iterative questioning. Open-text questions provide a unique challenge in automated systems because an administrator cannot anticipate them. For instance, the prior art allows for a selection of an industry from a drop-down list box containing a set of known values. Because all industries are known to the system, logic could be added to recommend a new question based upon the answer to the previous question.
  • a complex decision tree is created in advance, and the system is able to create a custom survey based upon the answers given by the user.
  • these systems are subject to the limitation of knowing all possible answers, and spending a significant amount of labor to program the decision tree to account for all possibilities. It also makes it very difficult to add new questions because the added responsibility of mapping each answer to the decision tree must be part of the process.
  • the invention described herein is not subject to such limitations because it allows the entry by users of any answer. Typically, this open-ended question is enabled by a text box with few restrictions on what can be entered, versus a drop down box, where a user must select one of the pre-defined answers.
  • the system allows open-ended text answers, a standard expert system is not possible because it is impossible to create a decision tree for an infinite number of questions that would make up a survey.
  • an artificial intelligence mechanism is usually employed to deal with the unknown variances in user input concerning which questions the users answer, as well as what those answers are.
  • the invention requires that a high degree of recall is maintained, which means a new user coming and using the system with the same set of profile data as an existing user, should have the same set of FieldGroups recommended in the same order (deterministic).
  • the system employs a system that defines relationships between FieldGroups, a minimum set of required FieldGroups, and an order that the FieldGroups appear during the survey. These system rules are combined with a set of weights and a subsystem which allows a qualified guess as to which FieldGroups should be suggested next, without requiring a predefined survey for each unique type of job profile.
  • the survey engine selects the most popular FieldGroup that the user has not yet answered, but that is related to the FieldGroups already posed to the user, and which is also contained in the profiles of the affinity group(s) to which the user currently belongs, subject to the level constraints and FieldGroup relationships described herein.
  • An administrator establishes the relationships and levels for FieldGroups, prior to a user completing the survey.
  • the system uses a collaborative filtering architecture for the survey engine to allow for a large number of FieldGroups and minimal administrative input. Although a collaborative filtering architecture is used, an alternative architecture such as a neural network might also be employed. Administrative tasks related to Suggesting FieldGroups are limited to cleaning input data that is used to teach the system, defining FieldGroup Relations, defining FieldGroup Levels, and defining affinity groups.
  • the inputs to the survey engine are a user's previous answers, affinity groups, relationships between FieldGroups (FieldGroup relations document), FieldGroup levels (or FieldGroup priority), and a triplet called “Popular FieldGroups,” that consists of an affinity group, a FieldGroup, and a weighted value.
  • the output for the survey engine is a single suggested FieldGroup.
  • the constraints in the suggested FieldGroup system consist of FieldGroup levels (or priorities) and defaults.
  • the survey wizard prioritizes the related FieldGroups by first selecting FieldGroups that possess the highest level (or priority) as defined by a level document for the particular wizard or broad goal. Each FieldGroup is assigned one and only one level per wizard.
  • Each level is assigned as a positive integer value to a FieldGroup, with lower levels constraining FieldGroups to appear earlier in the survey, and higher levels constraining the FieldGroup to appear later in the survey. This ensures that certain FieldGroups appear at or near the beginning of the survey, and other FieldGroups appear at or near the end. Additionally, the level document also groups FieldGroups together in each level, and assigns one FieldGroup from the group as a default FieldGroup, so that if an administrator wanted to ensure that at least one FieldGroup out of a group of FieldGroups is always asked, it is asked if no other FieldGroup from that level is asked.
  • Position may be a specialized type of job, which is appropriate to ask in certain industries, such as for a professional baseball player. In that case, job need not be asked because that information is captured in the position FieldGroup. For the most part, most profiles only need to answer job, whereas in specialized cases, they may need to answer alternate FieldGroups such as position. Therefore job and position are grouped; and job is set by an administrator as the default FieldGroup because the system requires that if people do not answer any other job-like question, they must answer job.
  • the weighted values on the Popular FieldGroups triplet are stored in a relational format.
  • the weights represent each FieldGroup's popularity. This is defined as the number of profiles for each affinity group that have answers for that FieldGroup.
  • Popularity is calculated by tallying up, for each FieldGroup, the number of users who have answered that FieldGroup, for each affinity group, and an association between an affinity, a FieldGroup, and a popularity, is stored in a relational table.
  • Constraints are generally applied to the system based upon naturally occurring relationships between FieldGroups, as they pertain to job profiles in general. For example, it is assumed that the existence of certain FieldGroups presupposes other FieldGroups. A human administrator with expertise in the domain defines these relationships based upon domain knowledge. Because the number of relations between FieldGroups is far fewer than the number of unique survey possibilities, it is efficient for an expert to define these relations, even if there are hundreds of FieldGroups. For instance, a FieldGroup such as “Bar Association” (generally used for lawyers and profiles requiring a legal degree) is only asked if the user's “Job,” or perhaps their “Industry” is known, but it is pointless to ask that FieldGroup if only the user's “Gender” is known.
  • Bar Association generally used for lawyers and profiles requiring a legal degree
  • FieldGroup such as “Bar Association” is related to “Job” and “Industry,” but not to “Gender.”
  • Relations such as these are referred to in the system as “FieldGroup Related,” and are stored in the system using a relational format. In addition to the levels, these relations act as constraints on the determination of the output of the suggest FieldGroup.
  • the survey engine selects amongst all the FieldGroups available for the survey, the FieldGroup that is the most popular (or highest weighted) FieldGroup among the affinity groups to which the user belongs, constrained by the levels and FieldGroup relations described previously. Finally, the FieldGroup is forwarded to the to the user for presentation. The user answers it, and the process is repeated for subsequent questions. If, after iteration, a FieldGroup no longer can be suggested that meets the criteria discussed herein, the system marks the users survey as complete and takes a pre-defined next action, such as showing a message to the user and moving on to the reporting aspects of the system.
  • FIGS. 9-10 show a non-limiting example of the system suggesting popular answers for a FieldGroup in the survey.
  • a mechanism exists in the system, which suggests possible answer choices for a particular FieldGroup, based upon several factors. These factors are, for example the FieldGroup being asked, an answer relations document, the user's profile and associated affinity groups, and a set of weights, which store the most popular answers for a particular FieldGroup.
  • the survey engine establishes a relationship between a particular FieldGroup and the suggested answers.
  • a particular FieldGroup is answer related to another FieldGroup if the answer to the first FieldGroup causes popular answers for the second FieldGroup to be suggested.
  • the ‘suggest popular answers’ algorithm is constrained by the answer relations, and the answer values are weighted using a table consisting of a list of associations between a value, an affinity group, and the number of profiles who have answered that particular value.
  • the survey engine selects the X most popular (most highly weighted) answers among the list, where X is an integer defined by an administrator as a reasonable number of values to display for a user to select from.
  • a survey begins with a FieldGroup for Industry, and the FieldGroup is displayed, which asks the user about the Industry they work in, and they respond “law,” a second FieldGroup for Law Firm, based on the process described earlier for suggested questions; is displayed and asks the user with which law firm they are associated.
  • the engine can also provide a list of law firms that were previously provided by other users to this FieldGroup, based on those who also identified themselves in the law Industry.
  • the system looks up all other FieldGroups that are answer-related to it.
  • the type of job (FieldGroup for Job) may be answer related to the affinity group associated with law firms, as may be the FieldGroup for “total hours billed per year,” etc. Therefore, because FieldGroup Job may be answer-related to FieldGroup Industry and FieldGroup Law Firm, the system suggests to the user to select his type of job from a list that includes corporate attorney, litigation attorney, paralegal, and so forth. While, if the user had answered “Computer Software” for the FieldGroup Industry, then the system would have suggested different possible answers for FieldGroup Job, such as computer programmer, senior software engineer, IT support technician, etc. Alternatively, instead of using the suggested popular answers, the user may enter a new value, and input a job title that is completely new to the system. Based on the user's responses, the system categorizes the users profile and aligns it with profiles that possess similar characteristics.
  • one aspect is to determine if the FieldGroup is an open FieldGroup, supporting open-text answers.
  • An open FieldGroup is one that allows for free text entries by a user, as opposed to a closed FieldGroup, where a user may only select values from a list or use numeric answers. If the FieldGroup is in an open format, the user may type a open-text answer, or select a suggested popular answer as described previously. When the user has made their entry, the system determines if the answer was typed as open-text or selected from the list of possible answers. If the answer is selected from a list of suggested answers, the users choice is saved in a database using the existing Answer ID for that existing answer in the system.
  • an algorithm is invoked to determine if the answer already exists in the database in a similar form to the users free-text entry.
  • possible existing alternatives from the database are suggested that possess similar characteristics as the free-text answer.
  • the alternative responses may possess different spelling of key words, grammatical variations, or combinations of other synonymous words.
  • the survey wizard tries to identify the underlying focus of the new answer and ensures that the system is gaining the right perspective from the user. If the user rejects the suggested alternatives, the system requests that the user retype the answer to confirm the response, and possibly enter more information about the answers, so that the system or an administrator may categorize the new answer. Once the user confirms the response by typing it a second time, the new answer is saved in the database and it becomes one of the possible answers that subsequent users may choose.
  • An administrator, or automated program can also verify that any new answers are qualified values, such as not containing swear-words or other anomalies.
  • An automated program may exist which uses a dictionary, or other software that can verify the validity of the value. If instead of a new open-text answer, the user selects one of the alternative responses, the selected answer is saved in the database using the existing Answer ID for that answer. Occasionally, the FieldGroup presented to the user is not of an open format, such as a small list of possible responses that do not change (are not open), or a numeric value, such as a salary figure, or a date, or Boolean values such as Yes/No. In numeric or date situations, there are bounds defined as well for answers.
  • the user's response is saved in the database. For instance, if the user is answering a date field, the date may be required to be within a certain range, or if a salary, it must be greater than ‘0.’ If the answer to this closed form of FieldGroup is not within the bounds of the FieldGroup definition, an error message is sent to the user asking for clarification or re-input of the answer.
  • the users profile is saved and cataloged in a database that allows the system to correlate it with other affinity groups and other user profiles to provide a comprehensive compensation report.
  • a major aspect of the invention concerns comparing an individual profile to all other profiles in the system, and determining which of these profiles are most similar.
  • the Profile Search algorithm returns a list of similar profiles. This similar list of profiles are then summarized and aggregated into a readable report that offers a complex analysis of a user's compensation and career opportunities.
  • This functionality allows the system to return an in-depth, customized report consisting of analysis of similar profiles in real time. It represents a more accurate picture of a user's compensation than a broad survey could, or one done by human analysts, or based only on job titles, or a standard expert system, etc.
  • the Profile Search algorithm runs, which returns a list of profiles to which the user's profile matches most closely to the search goal. Profiles are retrieved from the database as a set of FieldGroup,Field,AnswerValue triplets.
  • the system encapsulates and defines similarity between profiles by use of a scoring mechanism.
  • the scoring mechanism is generic in nature, and can apply to any attribute of a profile that has been defined in the system. It is important to note that different types of similarity can be established by the system. This allows different search goals to be executed. For example, a user may wish to know what projected salary they may expect to make in the marketplace, in their particular occupation, with the all other attributes, such as experience, location and industry, being similar. This is a common scenario for individuals wishing to switch jobs.
  • the invention supports this type of analysis using a pre-defined search goal document, which specifies that the user's profile is to be compared to a set of profiles that have similar jobs, in the same area, industry, etc.
  • a user may wish to know what other different jobs for which they may be qualified, given their skills, experience, location, etc. In this, case a different search goal is defined for this type of analysis.
  • FIG. 15 where a profile search scoring system is shown.
  • a search system is defined according to a relational structure, using a scoring and filtering system.
  • this scoring system both exact matches are considered, as well as affinity matches.
  • the affinity matches although scored lower than exact matches, are critical to the system, because they allow for a wide variety of similar attributes to be grouped together and compared in a more natural way.
  • the search goal definition for the search is retrieved. It specifies rules for how profile attributes are to the matched and compared.
  • the user's profile is then compared to all other profiles in the system, and each matching value from the profiles is assigned a score determined by the field which it matches on. For instance, exact matches on the FieldGroup job are assigned a certain relatively high score.
  • An affinity match i.e. “All People with Accounting Jobs,” is also considered because these matches are similar but not exactly the same. Because of this, these matches are given a slightly lower score.
  • Each profile is then score based on all of its affinity and value matches, and the score is totaled up by summing up the individual scores.
  • the FieldGroup for “Job” is just one kind.
  • other match scores can be defined for Skills, Specialties, and Certifications, and combined with scores for Job to create very accurate rankings of profiles returned in the search, to meet a given search goal.
  • a profile To be returned for a search (qualify), a profile must meet a certain overall threshold score, which is predefined by an administrator as part of the search goal document. Additionally, the profile must contain matches on specific deterministic fields. The threshold score removes any profiles that do not match on enough attributes to be considered a high quality match.
  • a profile matches on many non-deterministic values, such as gender, geographic location, or experience level, but does not match on something that is critical (or deterministic), such as job type or specialty.
  • An accountant may live next to an attorney, in the same age range, in the same community, who went to the same college, but they should not be considered matches because they are in different jobs and different job affinities. Therefore, in one type of search goal, only certain fields and affinities, such as those related to the job or specialty, are considered deterministic enough to be considered for matches.
  • Job and Skill are considered highly deterministic for basic compensation analysis
  • the system could be employed for other types of analysis where possibly other FieldGroups, such as Location or Gender, might be specified as highly deterministic instead. This allows for the definition and automation of many kinds of reports meeting many different profile use'search goals.
  • some combinations of attribute matches may be considered more valuable than other combinations. For instance, it may be preferable to match a smaller subset of profiles for which a location is closer to the target profile.
  • sets of matches can also be ranked. This is useful, for instance, in ranking profiles higher where geographic proximity is desirable.
  • the profile search system using matchgroups targets profiles within local regions first, ranking them higher than profiles that meet other match criteria but are in outlying regions. For instance, the search goal may first only find profiles in the same or surrounding cities. By defining metropolitan region affinity groups, one can find people in the same metropolitan region.
  • a set of profiles (usually but not necessarily retrieved via the profile search subsystem described elsewhere) is aggregated and their individual attributes are automatically summarized into calculations, such as averages, medians, standard deviations, and counts.
  • the calculations are aggregated for fast real-time retrieval.
  • Aggregate definitions define dimensions (such as skill) and measures (such as salary) into an aggregate (such as average salary by skill).
  • the system can summarize values into predetermined ranges, such as salary ranges or age ranges.
  • a report chart format defines how charts are displayed to an end user. Many different formats, such as HTML, PDF or JPEG, exist, and the report output format may be adjusted to work with any of these formats, and these charts may be displayed as bar charts, pie charts etc.
  • An administrator defines an aggregate definition, for example in an XML format, consisting of measure and a dimension and a name.
  • Each aggregate is calculated over an affinity group or groups. For example, after a profile search is executed, the list or resultant profiles is combined into an affinity group, such as “People Meeting Search Goal X for User Y,” and all available report definitions are executed, resulting in numerous aggregations.
  • Each aggregate definition must contain a measure or a dimension, or both. If no measure is specified, the aggregate is calculated as a count over the entire dimension. If no dimension is specified, the average, 25th, 50th, 75th median, standard deviation, and standard error, etc, are calculated over the entire population using standard algorithms. Aggregates for the affinity group can be calculated using this method. Using this method allows fast retrieval of aggregate information, and easy definition of new aggregations, which are available to any of thousands of affinity groups defined in the system.
  • the invention groups aggregations into charts and groups, and filters and arranges the charts in a layout which is understandable and useful to a human. Only charts that have enough data and only charts that are determined to be applicable to the user's profile are shown to the user. This allows for many reports that are only applicable to certain groups to be shown at any time, without having to predefine a report for any particular group. For instance, a group named “Pay Influencers” might contain the following charts.
  • Charts are grouped into logical sections that are recognizable to a human, and the system renders the charts in a grouped layout, so as to be in an understandable, cohesive, readable presentation.
  • charts that are defined according to a report specification.
  • a series of charts are combined into a group which is then parsed and formatted by a program into a user readable format, such as HTML.
  • a lawyer's profile may have matched to many other attorneys, all who have answered a FieldGroup for Bar Association and a FieldGroup for Hourly Billing Rate.
  • the compiled report for the lawyer can automatically shows a chart for “Average Billing Rate by Bar Association,” whereas a report for a High School Teachers would not show this chart because no matching profiles have either answered the “Bar Association” FieldGroup or the “Hourly Billing Rate” FieldGroup.
  • a rules engine is set up to process profile data automatically to search for common errors, problems, and faulty data.
  • the rules engine is an expert system defined by an expert on the domain. Because the survey system is an open system, which allows free-text user input, as well as freely input numeric data, rules are put in place to automatically monitor that data for validity. This helps to automate the process of data cleaning, and allows an administrator to review large numbers of new profiles more efficiently. For instance, a user may enter data that is obviously bogus, such a having a salary that is too high, or too low for the currency type and country, such as ‘0.’ Also, in some cases the rules engine may also automatically make changes to profile data.
  • the rules engine is implemented using a set of database queries as shown in FIG. 19 , and stored procedures that compare the profile to a set of predefined criteria, and then take some action as a result. Each of these queries is run against each new profile input into the system by users, and the results of the rules engine queries, including a list of changes determined by the rules, are stored in a table.
  • the rules engine allows stored procedures to be set up which can check for any type of data error.
  • Another example of a rule is if a user enters too many values for a particular FieldGroup, such as Skill. If a user has answered too many skills, chances are that corner of those skills are not valid or the user is just playing with the system, and hence, the profile is invalidated, and not considered for profile searches and reports.
  • a flag on the profile marks a profile as active or inactive. Inactive profiles are not used by any other part of the system for calculations, including comparisons in the survey engine, search goals, or reports. This way, data are cleaned and maintained as statistically valid. Many other flags are also set on a profile, and one skilled in the art can easily recognize that these flags are useful ways to gauge profiles' use in various aspects of the system. Rules in this form can also be used to modify existing profiles for data entry mistakes.
  • a user may commonly enter a new open-text value for one FieldGroup that is truly a value for another FieldGroup.
  • the rules engine moves the answer from one FieldGroup to another.
  • a rules engine may be extended to any situation that applies to a significant number of profiles.
  • an exemplary and non-limiting survey engine apparatus 110 (hereinafter “apparatus”) is shown, wherein an apparatus 110 is further connected to a plurality of users 120 by means of a network 130 .
  • a client 120 may be an individual requesting to participate in the survey for the purpose of establishing a personal ranking against other users who were surveyed using the apparatus 110 or other apparatus.
  • a user of a compensation survey implementation of an apparatus 110 such as discussed in more detail above, may wish to establish a prospective salary at a certain geographic area.
  • a client 120 for example client 120 - 1
  • a user may wish to receive a report of the average salary of a specific job function, or a report of the average salary expectation of same.
  • the Apparatus 110 is connected to the users 120 , for example client 120 - 1 , by means of a network 130 .
  • the network 130 may be, but is not limited to, a local area network (LAN), a wide area network (WAN), or the world-wide-web (WWW).
  • LAN local area network
  • WAN wide area network
  • WWW world-wide-web
  • a client 120 may easily access the apparatus 110 for both surveying and reporting purposes.
  • the apparatus 110 comprises a central processing unit (CPU) 112 , a work memory 114 , a code memory 116 , and a database storage 118 .
  • CPU central processing unit
  • the CPU 112 executes computer code stored, for example, in the code memory 116 , and which enables among others the handling of the communication protocols between the apparatus 110 and the clients 120 , the display of Web pages, search of the content of the database, preparation of reports, and the like.
  • Work memory 114 is used by the CPU 112 for its operation.
  • the database 118 retains user profiles, a plurality of questions and corresponding answers, report definitions, affinity groups, periodic reports, and the like.
  • the display components 210 are handled by the client 120 and the apparatus (or server) 110 .
  • the client 120 provides the display capabilities by using a Web browser 220 .
  • the apparatus 110 provides the display capabilities through the use of the hyper-text transfer protocol (HTTP) 230 .
  • Display components 210 enable the visual display of a survey and reports as may be required by a client 120 .
  • the system 100 further comprises business components 240 which have core components 245 and support components 250 .
  • Core components 245 comprise at least a survey engine 247 , a search engine 248 , and a reporting engine 246 .
  • Support components 250 may include, but are not limited to, commerce 251 , security 252 , administration 253 , and content management 254 .
  • Business components 240 enable the system 100 to provide the survey and reporting services in accordance with the disclosed invention.
  • the data components 260 are further comprised of a structured query language (SQL) server database 261 , a full text catalog 262 , and one or more extensible markup language (XML) configuration documents 263 . These components provide the data as may be required for the proper operation of the system 100 and particularly the apparatus 110 , in accordance with the disclosed invention.
  • SQL structured query language
  • XML extensible markup language
  • the apparatus 110 may reside on the apparatus 110 .
  • the database 118 may be distributed and accessible through the network 130 .
  • a plurality of apparatus 110 handle the surveys and have access to a plurality of respective databases 118 .
  • the code memory 116 contains, in addition-to other codes, a code that is targeted for the matching process required for the purpose of matching a user to a plurality of other users and establishing the user's unique affinity group.
  • the invention disclosed herein provides for a dynamic creation of affinity groups, in addition to the comparison to previously stored affinity groups.
  • the advantage of automatically detecting and creating affinity groups is in the fact the multiple parameters filled out by the user as part of the survey allow for increasingly accurately assessing the affinity group or groups with which a user should be associated with. A user of the system is therefore able to match his profile to that of others such as himself, and determine, for example, an expected salary for a certain job description or geographic area.
  • the method is not limited for job related surveys and these are provided merely as an exemplary and no-limiting usage scenarios.
  • a person skilled in the art would further note that the methods disclosed herein below may not necessarily require the exemplary apparatus disclosed above, and such methods may be applied to user profiles and affinity groups provided from other means, automated, manual, or combinations thereof.
  • a profile entered by a user is matched against a plurality of user profiles stored in the system. More specifically, the entered profile is matched to other profiles according to “matchgroups” (explained in more detail below) to determine if the profiles are similar, or like, each other. It is therefore possible to create a list of user profiles that are like the profile entered by the user, otherwise known as an affinity group.
  • matchgroups it is further possible to match a plurality of profiles against a goal, where a goal may be a set of parameters to which a subset of plurality of user profiles matches best according to a defined score and score threshold. It is further possible to match an entered profile to one or more affinity groups having known characteristics. In one embodiment of the invention such matching is performed at every stage of the answers provided by the user as the survey questions are presented and responded to. In another embodiment, such matching is applied after a completed survey, taking into account all attributes of the entered user profile, and producing a resulting group or groups of matching profiles according to the overall matching goal.
  • a matchgroup may be defined as a plurality of match fields and a plurality of affinity groups, each (match field, affinity group, and matchgroup) having a score value. If the entered profile has values matching other profiles' values according to the matchgroup, the matchgroup score is increased by the score value defined for the matching field. If the entered profile has values matching within an affinity group, the matchgroup score is increased by the affinity score value. The plurality of scores is tallied, allowing the determination of an overall score for the matchgroup. Furthermore, the process is repeated for any number of defined matchgroups having potentially different match fields and affinity groups, allowing the determination of an overall score for a defined goal. Matching profiles may be then included or excluded depending on threshold score values defined for this purpose, such as “Excellent Match,” “Good Match,” or “Bad Match.” The process of matching is hereinafter described in greater detail.
  • a user profile is received (hereinafter the “received user profile”). This may be a user profile that was prepared based on an automated survey system, discussed in more detail above.
  • a score is associated with each of the possible match groups as shown above.
  • a match group has a plurality of fields, affinity groups, and respective weights or scores, which allows for combinations of attribute matches, may be considered more valuable than other combinations. For instance, it may be preferable to match a smaller subset of profiles for which a location is closer to the target profile.
  • matches can also be ranked. This is useful, for instance, in ranking profiles higher where geographic proximity is desirable.
  • the profile search system using matchgroups targets profiles within local regions first, ranking them higher than profiles that meet other match criteria but are in outlying regions. For instance, the search goal may first only find profiles in the same or surrounding cities.
  • the search goal may first only find profiles in the same or surrounding cities.
  • metropolitan region affinity groups user profiles of people in the same metropolitan region can be found. However, there may not be enough valid data in the system to find user profiles in the same metropolitan region, and therefore, user profiles from a larger surrounding region, such as a state, or multi-state area, can be considered.
  • step S 330 a search is performed matching a plurality of user profiles to that of the received user profile.
  • a list of scores is returned for each of the compared user profiles. In one embodiment of the invention, the list is returned ordered from the highest to the lowest score.
  • any number of score fitness ranges is assigned, for example, a score range defining an excellent fit, a score range defining a good fit, and a score range defining below acceptable fit. More specifically, if the score is from ‘0’ to ‘100’ then excellent fit may be scores of ‘90’ through ‘100,’ good fit having scores of ‘70’ through ‘90,’ and so on.
  • step S 350 a report of the profiles best fitting the received user profile is provided, identifying those profiles that best fit as a new affinity group.
  • the created affinity group is checked against the already existing affinity groups and, if there is a match, no new affinity group is created.
  • An advantage of the disclosed invention over prior art is in the creation of an affinity group, which includes at least the received user profile. This allows for a larger match scope than the static assignment of an affinity group commonly found in prior art solutions.
  • a person skilled in the art may modify the disclosed flowchart to fit with the needs of specific programming languages, e.g. C++, or description languages, e.g. XML, without departing from the scope of the disclosed invention.
  • step S 330 of FIG. 3 for the profile search process is shown.
  • the flow chart explains the search and matching process of step S 330 discussed in FIG. 3 above.
  • a user profile to be matched to the user profile received in step S 310 is selected (hereinafter the “selected user profile”).
  • the selected user profile is one that was not previously selected during he specific execution of the current search and matchgroup.
  • the match group score of the selected user profile is determined in comparison with the received user profile.
  • the match score may be a binary value, for example, ‘0’ meaning no match and ‘1’ meaning match.
  • the specific match score is correlated further with the score value that is indicative of the weight a certain match field may have. As noted above, certain match fields may have a higher importance for the overall score than others, and hence have a higher score value, or effect, on the overall score result. It is further possible to limit the score, for example to a maximum score, for any matchgroup, so that a multitude of matches of a particular field do not skew the overall match score.
  • step S 330 - 30 it is checked if more match fields within the same user profile are to be checked and if so execution continues with step S 330 - 20 . Otherwise, execution continues with step S 330 - 40 .
  • step S 330 - 40 an overall fitness score respective of the user profile in comparison to the user profile received in step S 210 is calculated. This is a weighted value that takes into consideration all the relevant matchgroups. The overall result is used as a score indicative of the level of fitness between the two user profiles. In one embodiment of this invention these scores are normalized, for example, ‘100%’ meaning a best fit and ‘0%’ being no fit.
  • a matchgroup may be classified as being deterministic or non-deterministic, and requiring at least one deterministic match to exist within the overall score, thus allowing a more deterministic match than solely a weighted fuzzy type of result.
  • a matchgroup designated for ‘Job Type’ could be marked as deterministic, and then no profile is included in the overall affinity group unless there is at least a score, which may also require to be at least above a predefined threshold value, given to the Job Type matchgroup for the profile. Consequently, any score in a non-deterministic matchgroup could serve to increase the overall score of a profile, but in this case cannot by itself be sufficient to allow the profile to be included in the final fit.
  • parameters ‘such as this can be considered’ optional in different uses of the system, and are not meant to be limiting, but providing further flexibility.
  • a matchgroup can define a match field for Age Range, having defined ranges of Less than 17, 18-29, 30-45, 45-64, and 65+. If the entered profile matches by range with any user profile to be compared, the user profile could receive a score for such a matchgroup. It is also possible to score based on other Boolean criteria, such as a match field existing in a profile and having any value at all, as opposed to not existing in a profile. It is further possible to limit matches by a fixed criteria, such as only to allow for matches with profiles constrained to a particular arbitrary date range.
  • step S 330 - 50 it is checked whether there are more user profiles not yet matched against and if so execution continues with step S 330 - 10 . Otherwise, execution continues with step S 330 - 60 .
  • step S 330 - 60 a report including the score results of the user profiles to which the user profile received in step S 310 was matched against are provided. These results, for example, may be used for the purpose of reporting by the apparatus 110 discussed in more detail above.
  • the survey apparatus 110 uses this list to determine the affinity group to which a user belongs and to prepare tailored reports based, at least in part, on such an association to an affinity group or the determination of affinity groups as discussed in more detail above.
  • Lines 504 through 507 provide information of filters to be applied on the search data. This may include filters such as the state or a country in which a user is located or in which the user wishes to be. Another filter may be the profiles to be searched which can include all the profiles available, or a subset thereof. Lines 508 through 514 define sources for user profiles. Such sources may be available on database 118 . Such sources may be affinity groups created by apparatus 110 .
  • XML extensible markup language
  • Lines 516 - 522 , 523 - 527 , 528 - 532 , 533 - 537 , 538 - 541 , 542 - 548 , 549 - 552 , 553 - 557 , 558 - 561 , 562 - 564 , 565 - 567 , 568 - 577 , 578 - 581 , 582 - 585 , 586 - 590 , and 591 - 594 are examples of matches performed in accordance with the disclosed invention, as made applicable to an XML implementation.
  • Lines 595 - 598 define the ranking ranges so as to allow for the grouping of the matching profiles in accordance with the disclosed invention.
  • a person skilled in the art may modify the disclosed code to adapt to other types of searches for the purpose of matching user profiles without departing from the scope of the disclosed invention.
  • an exemplary and non-limiting flowchart 600 for the matching of a user profile to an affinity group is shown.
  • an affinity group to be matched to a user profile for example the user profile received in step S 310 shown in FIG. 3 .
  • This affinity group is one that was not matched before during the specific execution of the current search and match.
  • the match process entails the determination of scores of match groups of the affinity group with respect to the received profile.
  • a match group of the affinity group and the received user profile are compared to determine a score.
  • the score may be a binary value, for example, ‘0’ meaning low and ‘1’ meaning high.
  • step S 630 it is checked if more match groups between the affinity group and the received user profile are to be checked and if so execution continues with step S 620 . Otherwise, execution continues with step S 640 .
  • step S 640 an overall fitness score respective of the affinity group in comparison to the user profile is calculated.
  • step S 650 it is checked whether there are more affinity groups not: yet matched against and if so execution continues with step S 610 . Otherwise, execution continues with step S 660 .
  • step S 660 a report including the score results of the affinity groups to which the user profile was matched against are provided.
  • a person skilled in the art would be able to extend the apparatus 110 to provide the necessary reports by, for example, including the required code in code memory 116 .
  • a person skilled in the art may further modify the disclosed flowchart to fit with the needs of specific programming languages, e.g. C++, or description languages, e.g. XML, without departing from the scope of the disclosed invention.
  • step S 710 a user is asked if an extended report is desired and, if so, execution continues with step S 730 .
  • step S 720 a standard report is generated and, thereafter, execution continues with step S 750 .
  • the report is generated based on the user profile and the respective affinity group associated with that user profile generated by the matching system.
  • billing information is received from the user, for example client 120 - 1 .
  • Such information may include, but is not limited to, a purchase order (PO) number, a credit card number, or a debit card number.
  • PO purchase order
  • step S 740 an extended report is generated, the report being based on the user profile and the respective affinity group associated to that user profile.
  • An extended report may provide additional information otherwise not made available to the non-paying user.
  • such extended reports may include reports of a statistical nature respective of a plurality of users.
  • the extended report or the standard report is sent to the user, for example by means of displaying it as a Web page on the terminal of the client 120 , as an e-mail attachment, and the like.
  • the client 120 may further provides report parameters that influence the desired output of the report, for example, the type of information to be displayed, the manner in which such information is provided, and the like. Such information may be provided by means of defining goals as described in greater details in the above mentioned U.S. patent application.

Abstract

A method and apparatus for providing targeted online compensation reports that accounts for unique individual characteristics, such as related to a job, by using dynamic profiles is described. The preferred embodiment uses a survey engine (247) having a collaborative filtering engine that determines appropriate questions to ask the user during the survey, and may further provide suggested possible answers. A collection of user profiles are used for comparison purposes and to further produce individualized compensation reports.

Description

    BACKGROUND OF THE INVENTION FIELD OF INVENTION
  • The following disclosure relates generally to correlating statistical records and more particularly to correlating compensation records to unique individual profiles.
  • BACKGROUND OF THE INVENTION
  • Today, many reports are available that allow a user to find, read, purchase, or otherwise acquire reports on worker compensation. Most often these reports indicate average pay rates by industry, job type, locale, and sometimes they report more specific information about a particular industry or job, such as bonuses, stock options, average workweek, or immigration status, among other things. To create such compensation reports two approaches are typically used. One such approach is for a human analyst to research and find a statistically valid number of individuals with like characteristics, and devise a suite of compensation reports. This process is tedious, labor intensive and often expensive. For a truly detailed report the analyst must be relied on to do substantial investigation and synthesize and apply this information to the case at hand. Compensation consultants with years of experience and resources can generally accurately profile an individual's worth in the market place. However, such an analysis is extremely specialized and out of the reach of the typical consumer. Simpler and less costly reports are available but they are generally broadly classed and offer little utility.
  • Simpler and less costly reports are available but they are generally broadly classed and offer less utility. The majority- of software-based analysis provides a less expensive alternative but yields correspondingly limited information. Compensation services using current computer analysis programs generally gather data using some form of questionnaire and then feed the appropriate data into a computer database or spreadsheet. Or more typically, generalized data, such as from the US Bureau of Labor and Statistics, are used as a base and then extrapolated based on region and date, and often combined with third party surveys. Typically, a computer then is instructed to run an analysis of the data to provide statistical information such as averages, medians, and standard deviations on pre-determined groups of people. However, the information provided is not unique to an individual, but instead is a conglomeration of data that the program feels best represents the individual. Because the categorization of the individual is based solely on a limited, predetermined set of responses to the questionnaire, it offers little to no opportunity for evaluating unique characteristics. For example, an automated compensation service may categorize and calculate data showing that the average yearly salary of a “Computer Programmer Level 3” in Washington state is $64,250. This may or may not be applicable to a “Senior Application Software Engineer” with ten years of experience and special training in the skill C++, but because the closest answer describing the Senior Application Software Engineer's position in the initial survey was a “Computer Programmer Level 3,” the Senior Application Software Engineer has thus been categorized ineffectively, which removes any unique abilities he may possess. The Senior Application Software Engineer reading the aforementioned report cannot be sure how closely the published report figures apply to himself individually. There are a multitude of factors that affect any one individual's job compensation. The current generalized reporting methods for compensation reports cannot and do not incorporate factors that provide for an accurate job comparison and compensation analysis for individual users. Today's methods require the user to gauge or self-approximate themselves to a group of people being reported. Typically, such approximations are grouped by a specific job title that a human compensation analyst predetermines when creating a report or when designing a computer service that eventually generates the report. This grouping is generally not an exact match with the user's actual job title and responsibilities and often has little applicability to the users individual qualities. For example, the compensation analyst might have created a report for an isolated group called Computer Programmer Level 3. For individuals who possess the same characteristics as the data files used to create this group, the reports generated from such a compensation analysis are reasonably accurate. However, for individuals possessing unique capabilities, experiences, skills, or talents, the reports are essentially useless. The data are by definition misapplied because any differences in the compared data are arbitrarily reflected in the compensation report. This introduces doubt on the user's part as to how closely he can trust the report's applicability. Current compensation analysis techniques do not provide users with affordable, accurate, and personalized compensation reports. Job specific variables, critical to the accurate assessment of an individual's worth, are not correctly identified or uniformly applied. Furthermore, individuals within a particular field are unaware of the value of certain, often easily obtainable, qualifications. There is a need, therefore, for a system and method to provide online compensation reports using a more flexible survey system that produces dynamic profiles based on unique individual attributes and automated comparisons, and reports that account for these attributes.
  • SUMMARY OF THE INVENTION
  • The invention overcomes the limitations of the prior art and provides additional benefits. Under one aspect of the invention, profiles are used to produce individualized compensation reports. A survey engine is used to produce profiles of individuals that identify the individuals' unique characteristics. The survey engine incorporates a collaborative filtering engine that determines appropriate questions to ask the user during the survey, and also provides suggested possible answers. Additionally, the system allows for the use of open-text questions. Open-text questions allow for new answers to be input by the user, without the prior need for an administrator to pre-define the possible values for the system, as is typical in prior art. The system incorporates affinity groups around profile attributes (question answers), providing a basis for gauging similarity of profiles for various comparison and aggregation purposes described herein. A collaborative filtering engine is incorporated, which is both periodically modified by an administrator, and also tuned by users themselves based on their actions and responses. New affinity groups (associations of profiles) are incorporated by the survey engine to suggest new questions and possible answers in a survey. Additionally, some affinity groups are generated automatically by the system, and finally by users themselves to create new interesting relation'ships among profiles.
  • The collaborative filtering engine enables the capture of profile attributes that are targeted compensation variables concerning a profile. The system automatically incorporates new profiles into existing affinity groups, which allows more targeted survey questions and possible answers to be determined without requiring constant human training or intervention.
  • Without a collaborative filtering system, the system is unable to administer surveys accurately for users who do not fit into pre-defined categories, it is impossible to categorize every occupational variation because the system allows for open-text answers to questions, and because the system provides for different questions to be determined automatically and asked for differing types of user job profile, the collaborative filtering system, along with affinity groups and other requirements described herein, is employed. Because the collaborative filtering system allows for the system to make educated guesses within defined constraints, the system can handle new categorizations more effectively than a system wholly defined by a human administrator. Additionally, in this system, the administrator defines constraints that prohibit the survey from asking obviously wrong or out of place questions. An example of a constraint is requiring that if the user does not answer any of the suggested questions, a default question is always asked.
  • A new and surprising effect of using a collaborative filtering system to define a survey is that the system is accommodates a much larger population of data using a more targeted survey. Previous implementations of compensation surveys relied on a smaller sample size, a broad survey with generalized questions, or a larger base of analysts to design and conduct surveys and categorize the data. Thus, the survey was constrained purely by the human resources needed to conduct it. The invention described herein is not constrained as such and requires far fewer human resources to conduct such a detailed survey across many different job categories. Another key aspect of the invention includes the ability to search through the data that have been collected by the survey engine, using detailed search criteria defined by a search definition document. The document applies a scoring and filtering mechanism that returns the most appropriate set of profiles based on the user's goal for the analysis. The search document is useful because it allows an administrator to define natural relationships between profile attributes, such as those that apply routinely to the realm of compensation analysis of various classes of profiles. The document is then interpreted by a software algorithm, and used during the retrieval of relevant profiles, which can then be used for tallying and reporting. Multiple search definition documents can be created quickly, each one for a different goal. For example, one search goal may weight skills and certifications very highly. The results are useful for analyzing how to compare people with similar technical skills. Another search definition document weights experience and education higher, which is useful in seeing how a user compares to those profiles who have similar experience and education levels. A third example holds location constant, and compares a user to all other profiles with matching attributes in the same location.
  • Another aspect of the invention includes the automated ability to summarize and present the results of a profile search in a format that is useful to a human for compensation comparisons. A chart is defined as a series of values, such as skills, paired with a series of measures, such as average salary, median salary, standard deviation, etc. A sample chart is called “Average Salary By Skill” and it lists each Skill along with an associated average salary. A report is defined as a series of charts combined to provide an overall picture and analysis for a user. For instance, consider a report that aims to discover how a user compares with regard to skills and experience in similar jobs. This report incorporates many charts, as defined previously, which are combined into a format that gives a user a good analysis and understanding of the results of the user's search goal.
  • Prior art approaches are unable to automate the selection of charts within a compensation report, such as determining if “Average Salary by Practice Area” is applicable to a report presented for a particular user profile (it may only be applicable to a Lawyer, for instance). The prior art requires that the administrator know in advance, every chart that is relevant for a particular user. This shortcoming restricted previous inventions in one of two ways: limited the number of charts available, such that all charts were available to all compensation reports, regardless of their relevance, or they predefined different compensation reports by industry or job category, which would require a large amount of labor.
  • The invention described herein determines if a chart is relevant to a user based on the user's own profile, and displays it, and also only shows it if enough data exists for it to be valid statistically. There may be thousands of different charts of many different types and based on many different attributes and measures, and only a subset of them may apply to a particular users' compensation analysis. For instance, a lawyer may wish to see a report concerning “Average Bonus by Number of Hours Billed per Year,” but this report is not applicable to teachers or CEOs. In the prior art, each of these compensation reports is compiled by a human analyst, created at expense, or details are ignored, leaving only the most common charts in the compensation report, such as “Average Salary by Occupation and Location,” which are often the least useful to an individual trying to compare his compensation against his peers. The invention herein allows for a more targeted and relevant compensation report by focusing in an automated, scalable fashion, on attributes that are unique to a user, in addition to the attributes that are most common to all users.
  • One skilled in the art would also recognize that such a system could be used to match individual profiles to resumes because the profiles described herein are typically a subset of data gathered for a resume. For example, a user entering a profile into the system described herein could be easily matched to resumes using the same mechanisms employed to match against other profiles.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is an overview of a website implementation of the system according to the invention;
  • FIG. 2 is a general user flow through the system according to the invention;
  • FIG. 3 is a specific user flow for the first time a user accesses the system according to the invention;
  • FIG. 4 is a flow for the system to suggest a FieldGroup to a user according to the invention;
  • FIG. 5 is a flow for saving a user's answer to a database according to the invention;
  • FIG. 6 is an exemplary profile according to the invention;
  • FIG. 7 is an example of the system suggesting a FieldGroup in the survey
  • FIG. 8 is a continuation of FIG. 7
  • FIG. 9 is an example of the system suggesting popular answers for a FieldGroup, in the survey according to the invention;
  • FIG. 10 is a continuation of FIG. 9 according to the invention;
  • FIG. 11 is a flow for calculating and summarizing data according to the invention;
  • FIG. 12 is a flow for storing tokenized data in a database according to the invention;
  • FIG. 13 is a flow for populating an affinity group according to the invention;
  • FIG. 14 is a flow for generating a custom report for a user according to the invention;
  • FIG. 15 is an overview of the different aspects of a profile search according to the invention;
  • FIG. 16 is a flow for a profile search according to the invention;
  • FIG. 17 is an example of an affinity group according to the invention;
  • FIG. 18 is a flow for the rules engine according to the invention;
  • FIG. 19 is a series of rules used in the rules engine according to the invention;
  • FIG. 20 is an exemplary apparatus in accordance with the disclosed invention being further connected to a network according to the invention;
  • FIG. 21 is an exemplary flowchart for the purpose of creating a user specific affinity group according to the invention;
  • FIG. 22 is an exemplary flowchart showing the profile search process according to the invention;
  • FIG. 23 is an exemplary XML code for profile matching according to the invention;
  • FIG. 24 is an exemplary flowchart the matching of a user profile to an affinity group according to the invention; and
  • FIG. 25 is an exemplary flowchart for reporting of results of a matching process.
  • DETAILED DESCRIPTION OF THE INVENTION
  • A method and system for providing targeted online compensation reports that accounts for unique individual job characteristics by using dynamic profiles is described in detail herein. In the following description, numerous specific details are provided for survey flow, affinities, levels, suggest FieldGroup, suggest popular answers, save answers, profile search, scoring system, report aggregation, and rules engine. One skilled in the relevant art, however, will recognize that the invention can be practiced without one or more of the specific details, or with other symbols, methods, etc. In other instances, well-known methods or techniques are not shown:, or are not described in detail, to avoid obscuring aspects of the invention.
  • DEFINITIONS
  • Field—A single piece of information, corresponding to a particular question asked by the system. Examples, “City,” “State,” “Bonus Amount,” “Skill,” and “Job Title.”
  • FieldGroup—A set of related Fields that form a logical grouping of information into a single record. Also often referred to as a Question. Examples: “Salary” is a FieldGroup consisting of “Salary Amount,” “Currency,” and “Average Workweek.” “Job Location” is a FieldGroup consisting of “City,” “State,” and “Country.”
  • Survey—A set of FieldGroups asked in a particular order. An administrator may fix the order, or the system may determine the order using the “Suggest FieldGroup” algorithm defined below.
  • AnswerValue (also Value, Answer or Attribute)—A value (piece of information) corresponding to a Field, used as a response. Example: For field “City,” AnswerValues could be “Seattle,” “San Francisco,” “Miami,” or “Paris.”
  • Profile—A set of FieldGroups, Fields and AnswerValues that form a logical representation of an individual person or group of people. Example: “Industry=Finance, Job=Accountant, Employer-Name=Ernst & Young, Certification=CPA, Certification=CMA, Specialty=Taxation, Specialty=Cost Accounting, Years-In-Field=12, Salary-Amount=60,000, Salary-Currency-US Dollars, Salary-Workweek=40 Hours, Location-City=Baltimore, Location-State=Maryland, Location-Country=USA, School-Name=Princeton, School-Degree=Masters, School-DegreeYear=1995, Age=43, etc.”
  • Affinity Group (or Affinity)—A grouping of profiles that share common profile attributes, and further as shown in FIG. 17. Example: “All People who work in Law or Legal Professions.”
  • Affinity Definition—A Boolean representation of the common attributes shared by an affinity group. Example: For “All People who work in Law or Legal Professions”, the affinity definition might be: “Job=Lawyer OR Job=Attorney OR Job=District Court Judge or Industry=Legal OR Specialty=Trial Law OR etc.”
  • Profile Search—A detailed set of criteria used for matching profiles to other profiles and creating a scoring system that ranks the validity of the match.
  • Profile Search Document—A document that encompasses the criteria defined for the profile search.
  • Chart—An aggregation of data limited to an affinity group, in a format understandable to an end user. Example: Bar Chart for “Average Salary By Specialty.”
  • Report—A set of charts combined in a specific layout to provide a detailed analysis of a profile search and comparison goal.
  • Overview
  • Every individual possesses unique distinguishing characteristics in their employment profile. These unique characteristics, even when seemingly minor, can correspond to differences in employment compensation. Being able to have a custom report that compares relevant characteristics for each individual, and having an understanding of what the market is willing to pay for the individual's abilities based on such comparison, is an important step in finding and effectively negotiating employment opportunities, as well as making informed career decisions. The method and apparatus presented herein provide comparative compensation reports based on characteristics determined through a survey and a scored-attribute-matching search and reporting process.
  • Reference is now made to FIG. 2 where a general user flow through the system is shown. A user begins the investigation of their worth by accessing an Internet Website through a user interface, such as a personal computer, personal data assistant, or similar device. Once access has been established, the user conveys to the system his desired compensation comparison objectives/goals, as shown in FIG. 1. Once the objectives have been identified a survey engine, also referred to as survey wizard, begins identifying the unique characteristics of the user that are most applicable to determining a compensation level. The survey engine uses a combination of open and closed FieldGroups to create an individual profile. An open FieldGroup is defined as having at least one Field for which the user may enter an answer in free text. A closed FieldGroup is defined as having a Fields for which the user may only select from a list of predefined choices. After each FieldGroup is answered during the survey process, the individual profile is associated with one or more affinity groups. The affinity groups are discrete groupings of individual profiles based on Boolean criteria. The Boolean criteria, either simple or complex, are based on a combination of values from one or more attributes on a profile, such as job, skill, location, etc. A user's profile attributes are compared against each affinity definition, and if the profile attribute meets the affinity definition criteria, the user is considered to be a member of that affinity group. For example, a user may be considered part of the affinity group “People in Information Technology/Computer Networking” if they have answered having certain skills, such as Skill=TCP/IP, Skill=Cisco Routers, Skill=Windows NT Networking, or Certification=Microsoft Certified Systems Engineer.
  • The survey engine, or system, discussed in more detail below, through iteration, asks FieldGroups of the user until the engine determines no more FieldGroups should be asked. Upon beginning the survey process, the user's objectives/goals are confirmed through an initial set of questions, as shown in FIG. 1. The goal establishes broad areas that should be investigated by the survey engine. The engine, using this goal, suggests the next FieldGroup (Question) to be posed to the user, from the set of all available FieldGroups in the system shown in FIG. 4. The FieldGroup along, with possible popular answers obtained from previous questioning of different users, is presented to the user. The system examines the answer offered by the user and, if appropriate, saves it in a database, as shown in FIG. 5. This process continues until the engine determines that the desired goal has been ascertained and no more FieldGroups need to be asked. The engine queries if any more goals need to be examined returning the process to establishing a goal for Questioning. The process continues as described herein until profile data for all the applicable goals are created and the survey is complete.
  • The system uses affinities or affinity groups to categorize and group users in many segments of the application. An affinity group is defined as a group of profiles defined by a set of profile criteria, called an affinity definition. Affinity definitions are a combination of values from fields defined in the system also shown in FIG. 6. For example, Job=Human Resource Manager is an affinity definition for the affinity group consisting of all the profiles such that users answered Job=Human Resources Manager. Criteria can be logically defined to create more complex criteria, using standard Boolean operators such as AND and OR. For example, the affinity group called “San Francisco Java Programmers” might have the following affinity definition: “City=San Francisco AND (Job=Software Programmer or Job=Software Developer or Job=Web Developer) AND Skill=Java.” The corresponding set of profiles that match this definition is the affinity group. The affinity definition is stored in a relational format, which is easily and quickly retrievable, and searchable. One skilled in the art would recognize that such a definition could be stored in a wide variety of formats suitable to the task at hand, or modified for improved CPU performance. A program quickly compares the user's profile to all affinity definitions stored in the system, and determines to which affinity groups a user belongs. One skilled in the art would be able to recognize that more complicated affinities are possible, using more complex Boolean logic such as NOT, XOR, etc. Additionally, affinities can be created-which group numeric data by ranges. For instance, all people in a certain age range could be grouped together. Affinities are associations of data, defined by a human, an administrator, or users, which allow the system to create a more intelligent output for use by many of the sub-systems described herein. Additional detail is provided below in the form of flowcharts.
  • The survey engine is required to ask FieldGroups that are relevant for a particular user. Many different job profiles have different attributes that affect one's compensation. For example, a CEO may need an answer to the FieldGroup (question) Company Revenue in his profile, while a Lawyer might need Practice Area or Bar Association Memberships. The suggest FieldGroup algorithm is designed to ask pertinent questions of the user depending on the user's compensation analysis goal and the type of profile the user has, learned through iterative questioning. Open-text questions provide a unique challenge in automated systems because an administrator cannot anticipate them. For instance, the prior art allows for a selection of an industry from a drop-down list box containing a set of known values. Because all industries are known to the system, logic could be added to recommend a new question based upon the answer to the previous question. In such a system, a complex decision tree is created in advance, and the system is able to create a custom survey based upon the answers given by the user. However, these systems are subject to the limitation of knowing all possible answers, and spending a significant amount of labor to program the decision tree to account for all possibilities. It also makes it very difficult to add new questions because the added responsibility of mapping each answer to the decision tree must be part of the process. The invention described herein is not subject to such limitations because it allows the entry by users of any answer. Typically, this open-ended question is enabled by a text box with few restrictions on what can be entered, versus a drop down box, where a user must select one of the pre-defined answers.
  • Because the system allows open-ended text answers, a standard expert system is not possible because it is impossible to create a decision tree for an infinite number of questions that would make up a survey. In such situations in other art, an artificial intelligence mechanism is usually employed to deal with the unknown variances in user input concerning which questions the users answer, as well as what those answers are. However, the invention requires that a high degree of recall is maintained, which means a new user coming and using the system with the same set of profile data as an existing user, should have the same set of FieldGroups recommended in the same order (deterministic). To accommodate these requirements, ie. both recall and variability, the system employs a system that defines relationships between FieldGroups, a minimum set of required FieldGroups, and an order that the FieldGroups appear during the survey. These system rules are combined with a set of weights and a subsystem which allows a qualified guess as to which FieldGroups should be suggested next, without requiring a predefined survey for each unique type of job profile.
  • To pick the next FieldGroup to be asked of the user, the survey engine selects the most popular FieldGroup that the user has not yet answered, but that is related to the FieldGroups already posed to the user, and which is also contained in the profiles of the affinity group(s) to which the user currently belongs, subject to the level constraints and FieldGroup relationships described herein. An administrator establishes the relationships and levels for FieldGroups, prior to a user completing the survey. As explained earlier, the system uses a collaborative filtering architecture for the survey engine to allow for a large number of FieldGroups and minimal administrative input. Although a collaborative filtering architecture is used, an alternative architecture such as a neural network might also be employed. Administrative tasks related to Suggesting FieldGroups are limited to cleaning input data that is used to teach the system, defining FieldGroup Relations, defining FieldGroup Levels, and defining affinity groups.
  • The inputs to the survey engine are a user's previous answers, affinity groups, relationships between FieldGroups (FieldGroup relations document), FieldGroup levels (or FieldGroup priority), and a triplet called “Popular FieldGroups,” that consists of an affinity group, a FieldGroup, and a weighted value. The output for the survey engine is a single suggested FieldGroup. The constraints in the suggested FieldGroup system consist of FieldGroup levels (or priorities) and defaults. The survey wizard prioritizes the related FieldGroups by first selecting FieldGroups that possess the highest level (or priority) as defined by a level document for the particular wizard or broad goal. Each FieldGroup is assigned one and only one level per wizard. Each level is assigned as a positive integer value to a FieldGroup, with lower levels constraining FieldGroups to appear earlier in the survey, and higher levels constraining the FieldGroup to appear later in the survey. This ensures that certain FieldGroups appear at or near the beginning of the survey, and other FieldGroups appear at or near the end. Additionally, the level document also groups FieldGroups together in each level, and assigns one FieldGroup from the group as a default FieldGroup, so that if an administrator wanted to ensure that at least one FieldGroup out of a group of FieldGroups is always asked, it is asked if no other FieldGroup from that level is asked. For instance, there may be two separate FieldGroups, “Job” and “Position.” Position may be a specialized type of job, which is appropriate to ask in certain industries, such as for a professional baseball player. In that case, job need not be asked because that information is captured in the position FieldGroup. For the most part, most profiles only need to answer job, whereas in specialized cases, they may need to answer alternate FieldGroups such as position. Therefore job and position are grouped; and job is set by an administrator as the default FieldGroup because the system requires that if people do not answer any other job-like question, they must answer job.
  • The weighted values on the Popular FieldGroups triplet are stored in a relational format. The weights represent each FieldGroup's popularity. This is defined as the number of profiles for each affinity group that have answers for that FieldGroup. Popularity is calculated by tallying up, for each FieldGroup, the number of users who have answered that FieldGroup, for each affinity group, and an association between an affinity, a FieldGroup, and a popularity, is stored in a relational table. By this mechanism, a feedback loop is created between the user's profile questions and answers, and the survey wizard. As user profiles are entered into the system via the survey process, new associations of similar profiles are produced and those results are integrated into the popularity of each corresponding Fieldgroup, yielding a subsequently more and more precise survey for differing types of job profiles. Additional flowchart discussion is provided below.
  • Constraints are generally applied to the system based upon naturally occurring relationships between FieldGroups, as they pertain to job profiles in general. For example, it is assumed that the existence of certain FieldGroups presupposes other FieldGroups. A human administrator with expertise in the domain defines these relationships based upon domain knowledge. Because the number of relations between FieldGroups is far fewer than the number of unique survey possibilities, it is efficient for an expert to define these relations, even if there are hundreds of FieldGroups. For instance, a FieldGroup such as “Bar Association” (generally used for lawyers and profiles requiring a legal degree) is only asked if the user's “Job,” or perhaps their “Industry” is known, but it is pointless to ask that FieldGroup if only the user's “Gender” is known. As such, a FieldGroup such as “Bar Association” is related to “Job” and “Industry,” but not to “Gender.” Relations such as these are referred to in the system as “FieldGroup Related,” and are stored in the system using a relational format. In addition to the levels, these relations act as constraints on the determination of the output of the suggest FieldGroup.
  • Given the constraints and weights that are resident in the system, the survey engine selects amongst all the FieldGroups available for the survey, the FieldGroup that is the most popular (or highest weighted) FieldGroup among the affinity groups to which the user belongs, constrained by the levels and FieldGroup relations described previously. Finally, the FieldGroup is forwarded to the to the user for presentation. The user answers it, and the process is repeated for subsequent questions. If, after iteration, a FieldGroup no longer can be suggested that meets the criteria discussed herein, the system marks the users survey as complete and takes a pre-defined next action, such as showing a message to the user and moving on to the reporting aspects of the system.
  • Reference is now made to FIGS. 9-10 that show a non-limiting example of the system suggesting popular answers for a FieldGroup in the survey. A mechanism exists in the system, which suggests possible answer choices for a particular FieldGroup, based upon several factors. These factors are, for example the FieldGroup being asked, an answer relations document, the user's profile and associated affinity groups, and a set of weights, which store the most popular answers for a particular FieldGroup. Just as there is a relational basis between FieldGroups asked during the survey process, one aspect of the survey engine establishes a relationship between a particular FieldGroup and the suggested answers. A particular FieldGroup is answer related to another FieldGroup if the answer to the first FieldGroup causes popular answers for the second FieldGroup to be suggested.
  • The ‘suggest popular answers’ algorithm is constrained by the answer relations, and the answer values are weighted using a table consisting of a list of associations between a value, an affinity group, and the number of profiles who have answered that particular value. Among the constrained answers, the survey engine selects the X most popular (most highly weighted) answers among the list, where X is an integer defined by an administrator as a reasonable number of values to display for a user to select from.
  • For example if a survey begins with a FieldGroup for Industry, and the FieldGroup is displayed, which asks the user about the Industry they work in, and they respond “law,” a second FieldGroup for Law Firm, based on the process described earlier for suggested questions; is displayed and asks the user with which law firm they are associated. Based on the Suggest Popular Answers algorithm, the engine can also provide a list of law firms that were previously provided by other users to this FieldGroup, based on those who also identified themselves in the law Industry. The affinity group Industry=“law” is, therefore, answer related to the FieldGroup for Law Firm because the suggested answers for the FieldGroup Law Firm are related to the answer provided by the first FieldGroup Industry. Correspondingly, for the FieldGroup Law Firm, the system looks up all other FieldGroups that are answer-related to it. For example, the type of job (FieldGroup for Job) may be answer related to the affinity group associated with law firms, as may be the FieldGroup for “total hours billed per year,” etc. Therefore, because FieldGroup Job may be answer-related to FieldGroup Industry and FieldGroup Law Firm, the system suggests to the user to select his type of job from a list that includes corporate attorney, litigation attorney, paralegal, and so forth. While, if the user had answered “Computer Software” for the FieldGroup Industry, then the system would have suggested different possible answers for FieldGroup Job, such as computer programmer, senior software engineer, IT support technician, etc. Alternatively, instead of using the suggested popular answers, the user may enter a new value, and input a job title that is completely new to the system. Based on the user's responses, the system categorizes the users profile and aligns it with profiles that possess similar characteristics.
  • When the survey wizard displays a question, one aspect is to determine if the FieldGroup is an open FieldGroup, supporting open-text answers. An open FieldGroup is one that allows for free text entries by a user, as opposed to a closed FieldGroup, where a user may only select values from a list or use numeric answers. If the FieldGroup is in an open format, the user may type a open-text answer, or select a suggested popular answer as described previously. When the user has made their entry, the system determines if the answer was typed as open-text or selected from the list of possible answers. If the answer is selected from a list of suggested answers, the users choice is saved in a database using the existing Answer ID for that existing answer in the system. If the answer was free/open-text, an algorithm is invoked to determine if the answer already exists in the database in a similar form to the users free-text entry. Using search technology, possible existing alternatives from the database are suggested that possess similar characteristics as the free-text answer. The alternative responses may possess different spelling of key words, grammatical variations, or combinations of other synonymous words. By doing this, the survey wizard tries to identify the underlying focus of the new answer and ensures that the system is gaining the right perspective from the user. If the user rejects the suggested alternatives, the system requests that the user retype the answer to confirm the response, and possibly enter more information about the answers, so that the system or an administrator may categorize the new answer. Once the user confirms the response by typing it a second time, the new answer is saved in the database and it becomes one of the possible answers that subsequent users may choose.
  • An administrator, or automated program, can also verify that any new answers are qualified values, such as not containing swear-words or other anomalies. An automated program may exist which uses a dictionary, or other software that can verify the validity of the value. If instead of a new open-text answer, the user selects one of the alternative responses, the selected answer is saved in the database using the existing Answer ID for that answer. Occasionally, the FieldGroup presented to the user is not of an open format, such as a small list of possible responses that do not change (are not open), or a numeric value, such as a salary figure, or a date, or Boolean values such as Yes/No. In numeric or date situations, there are bounds defined as well for answers. If the answer to the FieldGroup fits into the bounds of the FieldGroup definition, the user's response is saved in the database. For instance, if the user is answering a date field, the date may be required to be within a certain range, or if a salary, it must be greater than ‘0.’ If the answer to this closed form of FieldGroup is not within the bounds of the FieldGroup definition, an error message is sent to the user asking for clarification or re-input of the answer. Ultimately, the users profile is saved and cataloged in a database that allows the system to correlate it with other affinity groups and other user profiles to provide a comprehensive compensation report.
  • Reference is now made to FIG. 16, where a flow for a profile search is shown. A major aspect of the invention concerns comparing an individual profile to all other profiles in the system, and determining which of these profiles are most similar. The Profile Search algorithm returns a list of similar profiles. This similar list of profiles are then summarized and aggregated into a readable report that offers a complex analysis of a user's compensation and career opportunities. This functionality allows the system to return an in-depth, customized report consisting of analysis of similar profiles in real time. It represents a more accurate picture of a user's compensation than a broad survey could, or one done by human analysts, or based only on job titles, or a standard expert system, etc. The Profile Search algorithm runs, which returns a list of profiles to which the user's profile matches most closely to the search goal. Profiles are retrieved from the database as a set of FieldGroup,Field,AnswerValue triplets. The system encapsulates and defines similarity between profiles by use of a scoring mechanism. The scoring mechanism is generic in nature, and can apply to any attribute of a profile that has been defined in the system. It is important to note that different types of similarity can be established by the system. This allows different search goals to be executed. For example, a user may wish to know what projected salary they may expect to make in the marketplace, in their particular occupation, with the all other attributes, such as experience, location and industry, being similar. This is a common scenario for individuals wishing to switch jobs. The invention supports this type of analysis using a pre-defined search goal document, which specifies that the user's profile is to be compared to a set of profiles that have similar jobs, in the same area, industry, etc. In another variation, a user may wish to know what other different jobs for which they may be qualified, given their skills, experience, location, etc. In this, case a different search goal is defined for this type of analysis.
  • Reference is now made to FIG. 15, where a profile search scoring system is shown. To encapsulate each type of search goal, a search system is defined according to a relational structure, using a scoring and filtering system. In this scoring system, both exact matches are considered, as well as affinity matches. The affinity matches, although scored lower than exact matches, are critical to the system, because they allow for a wide variety of similar attributes to be grouped together and compared in a more natural way. This solves a critical problem in this area of invention, i.e., previous incarnations of occupational comparison systems require exact matches, and hence, lose much of their value, because a large percentage of real world profiles that are similar do not usually have values with exact matches, or the number of true variations is much higher than is usually accounted for in fixed reporting systems. For instance, an IRS autitor may wish to be compared to an accountant, but because they have different job titles, they may never be compared in other systems. In this implementation, affinities (as described elsewhere) are constantly being added and updated through manual and automated processes, which can link these values together, and hence, return similar profiles.
  • When a profile search is requested, the search goal definition for the search is retrieved. It specifies rules for how profile attributes are to the matched and compared. The user's profile is then compared to all other profiles in the system, and each matching value from the profiles is assigned a score determined by the field which it matches on. For instance, exact matches on the FieldGroup job are assigned a certain relatively high score. An affinity match i.e. “All People with Accounting Jobs,” is also considered because these matches are similar but not exactly the same. Because of this, these matches are given a slightly lower score. Each profile is then score based on all of its affinity and value matches, and the score is totaled up by summing up the individual scores. One skilled in the art can understand that many different types of search goals and match criteria can be defined, depending on the type of FieldGroup. The FieldGroup for “Job” is just one kind. For example, other match scores can be defined for Skills, Specialties, and Certifications, and combined with scores for Job to create very accurate rankings of profiles returned in the search, to meet a given search goal. To be returned for a search (qualify), a profile must meet a certain overall threshold score, which is predefined by an administrator as part of the search goal document. Additionally, the profile must contain matches on specific deterministic fields. The threshold score removes any profiles that do not match on enough attributes to be considered a high quality match. For instance, it is possible that a profile matches on many non-deterministic values, such as gender, geographic location, or experience level, but does not match on something that is critical (or deterministic), such as job type or specialty. An accountant may live next to an attorney, in the same age range, in the same community, who went to the same college, but they should not be considered matches because they are in different jobs and different job affinities. Therefore, in one type of search goal, only certain fields and affinities, such as those related to the job or specialty, are considered deterministic enough to be considered for matches. Again, one skilled in the art will understand that while Job and Skill are considered highly deterministic for basic compensation analysis, the system could be employed for other types of analysis where possibly other FieldGroups, such as Location or Gender, might be specified as highly deterministic instead. This allows for the definition and automation of many kinds of reports meeting many different profile use'search goals.
  • Additionally, some combinations of attribute matches may be considered more valuable than other combinations. For instance, it may be preferable to match a smaller subset of profiles for which a location is closer to the target profile. By employing matchgroups within the profile search goal document, sets of matches can also be ranked. This is useful, for instance, in ranking profiles higher where geographic proximity is desirable. By creating a set of affinities (described elsewhere) that group together profiles by regions, the profile search system using matchgroups targets profiles within local regions first, ranking them higher than profiles that meet other match criteria but are in outlying regions. For instance, the search goal may first only find profiles in the same or surrounding cities. By defining metropolitan region affinity groups, one can find people in the same metropolitan region. However, there may not be enough valid data in the system to find profiles in the same metropolitan region, and therefore, one must also consider profiles from a larger surrounding region, such as a state, or multi-state area, and choose them if necessary. By including affinities in a ranked fashion in matchgroups, it is possible to return profiles that are closest to our own by having them rank the highest. One skilled in the art would recognize that closeness could apply to any attribute of a profile in addition to location, such as experience level or age ranges.
  • Reference is now made to FIG. 14 where another aspect of the invention, a set of profiles (usually but not necessarily retrieved via the profile search subsystem described elsewhere) is aggregated and their individual attributes are automatically summarized into calculations, such as averages, medians, standard deviations, and counts. The calculations are aggregated for fast real-time retrieval. Aggregate definitions define dimensions (such as skill) and measures (such as salary) into an aggregate (such as average salary by skill). Additionally, the system can summarize values into predetermined ranges, such as salary ranges or age ranges. A report chart format defines how charts are displayed to an end user. Many different formats, such as HTML, PDF or JPEG, exist, and the report output format may be adjusted to work with any of these formats, and these charts may be displayed as bar charts, pie charts etc.
  • An administrator defines an aggregate definition, for example in an XML format, consisting of measure and a dimension and a name. Each aggregate is calculated over an affinity group or groups. For example, after a profile search is executed, the list or resultant profiles is combined into an affinity group, such as “People Meeting Search Goal X for User Y,” and all available report definitions are executed, resulting in numerous aggregations. Each aggregate definition must contain a measure or a dimension, or both. If no measure is specified, the aggregate is calculated as a count over the entire dimension. If no dimension is specified, the average, 25th, 50th, 75th median, standard deviation, and standard error, etc, are calculated over the entire population using standard algorithms. Aggregates for the affinity group can be calculated using this method. Using this method allows fast retrieval of aggregate information, and easy definition of new aggregations, which are available to any of thousands of affinity groups defined in the system.
  • Because hundreds of aggregate definitions are available over thousands of affinity groups, it is impossible to present all of this data to a human in a format which they could easily and quickly understand because the resulting data would consume thousands of pages. To solve this problem, the invention groups aggregations into charts and groups, and filters and arranges the charts in a layout which is understandable and useful to a human. Only charts that have enough data and only charts that are determined to be applicable to the user's profile are shown to the user. This allows for many reports that are only applicable to certain groups to be shown at any time, without having to predefine a report for any particular group. For instance, a group named “Pay Influencers” might contain the following charts. “Average Salary by Job,” “Average Salary By Practice Area,” “Average Salary By Teaching Rank,” “Average Salary By Skill,” “Average Salary By Experience,” “Average Billing Rate by Bar Association,” etc. But not all of the charts in this group are displayed, depending on the applicability to the user's profile. For example, the chart “Average Salary By Teaching Rank” is not displayed if the report is for a lawyer's profile. Another group named “Geographic Outlook” might contain charts such as “Average Salary By City,” or “Average Salary By State.”
  • Charts are grouped into logical sections that are recognizable to a human, and the system renders the charts in a grouped layout, so as to be in an understandable, cohesive, readable presentation. For a particular set of similar profiles, there are many charts that are defined according to a report specification. A series of charts are combined into a group which is then parsed and formatted by a program into a user readable format, such as HTML.
  • For instance, a lawyer's profile may have matched to many other attorneys, all who have answered a FieldGroup for Bar Association and a FieldGroup for Hourly Billing Rate. The compiled report for the lawyer can automatically shows a chart for “Average Billing Rate by Bar Association,” whereas a report for a High School Teachers would not show this chart because no matching profiles have either answered the “Bar Association” FieldGroup or the “Hourly Billing Rate” FieldGroup. This is an important innovation over previous approaches, which are restricted to returning generic assessments that group individuals into large, often useless, categorizations, ignoring subtler, yet far more useful categorizations.
  • Reference is now made to FIG. 18 where a flow for the rules engine is shown. A rules engine is set up to process profile data automatically to search for common errors, problems, and faulty data. The rules engine is an expert system defined by an expert on the domain. Because the survey system is an open system, which allows free-text user input, as well as freely input numeric data, rules are put in place to automatically monitor that data for validity. This helps to automate the process of data cleaning, and allows an administrator to review large numbers of new profiles more efficiently. For instance, a user may enter data that is obviously bogus, such a having a salary that is too high, or too low for the currency type and country, such as ‘0.’ Also, in some cases the rules engine may also automatically make changes to profile data.
  • The rules engine is implemented using a set of database queries as shown in FIG. 19, and stored procedures that compare the profile to a set of predefined criteria, and then take some action as a result. Each of these queries is run against each new profile input into the system by users, and the results of the rules engine queries, including a list of changes determined by the rules, are stored in a table. The rules engine allows stored procedures to be set up which can check for any type of data error.
  • Another example of a rule is if a user enters too many values for a particular FieldGroup, such as Skill. If a user has answered too many skills, chances are that corner of those skills are not valid or the user is just playing with the system, and hence, the profile is invalidated, and not considered for profile searches and reports. A flag on the profile marks a profile as active or inactive. Inactive profiles are not used by any other part of the system for calculations, including comparisons in the survey engine, search goals, or reports. This way, data are cleaned and maintained as statistically valid. Many other flags are also set on a profile, and one skilled in the art can easily recognize that these flags are useful ways to gauge profiles' use in various aspects of the system. Rules in this form can also be used to modify existing profiles for data entry mistakes. For instance, a user may commonly enter a new open-text value for one FieldGroup that is truly a value for another FieldGroup. To keep data as valid as possible in an open system such as this, the rules engine moves the answer from one FieldGroup to another. One can easily see that a rules engine may be extended to any situation that applies to a significant number of profiles.
  • Referring now to FIG. 20 an exemplary and non-limiting survey engine apparatus 110 (hereinafter “apparatus”) is shown, wherein an apparatus 110 is further connected to a plurality of users 120 by means of a network 130. A client 120, for example client 120-1, may be an individual requesting to participate in the survey for the purpose of establishing a personal ranking against other users who were surveyed using the apparatus 110 or other apparatus. For example, a user of a compensation survey implementation of an apparatus 110, such as discussed in more detail above, may wish to establish a prospective salary at a certain geographic area. In another embodiment of the invention, a client 120, for example client 120-1, may be interested in receiving reports providing statistical information of a plurality of users who have previously answered the survey. For example, in the case of a compensation survey implementation of the apparatus 110, such a user may wish to receive a report of the average salary of a specific job function, or a report of the average salary expectation of same.
  • The Apparatus 110 is connected to the users 120, for example client 120-1, by means of a network 130. The network 130 may be, but is not limited to, a local area network (LAN), a wide area network (WAN), or the world-wide-web (WWW). Through the use of Web pages and standard communication protocols, a client 120 may easily access the apparatus 110 for both surveying and reporting purposes. In one embodiment, the apparatus 110 comprises a central processing unit (CPU) 112, a work memory 114, a code memory 116, and a database storage 118. The CPU 112 executes computer code stored, for example, in the code memory 116, and which enables among others the handling of the communication protocols between the apparatus 110 and the clients 120, the display of Web pages, search of the content of the database, preparation of reports, and the like. Work memory 114 is used by the CPU 112 for its operation. The database 118 retains user profiles, a plurality of questions and corresponding answers, report definitions, affinity groups, periodic reports, and the like.
  • Reference is now made again to FIG. 1, where the details of a Website implementation of the system 100 incorporating the apparatus 110 are further discussed. There are three basic building blocks to the system: display components 210, business components 240, and data components 260. The display components 210 are handled by the client 120 and the apparatus (or server) 110. The client 120 provides the display capabilities by using a Web browser 220. The apparatus 110 provides the display capabilities through the use of the hyper-text transfer protocol (HTTP) 230. Display components 210 enable the visual display of a survey and reports as may be required by a client 120. The system 100 further comprises business components 240 which have core components 245 and support components 250. Core components 245 comprise at least a survey engine 247, a search engine 248, and a reporting engine 246. Support components 250 may include, but are not limited to, commerce 251, security 252, administration 253, and content management 254. Business components 240 enable the system 100 to provide the survey and reporting services in accordance with the disclosed invention. The data components 260 are further comprised of a structured query language (SQL) server database 261, a full text catalog 262, and one or more extensible markup language (XML) configuration documents 263. These components provide the data as may be required for the proper operation of the system 100 and particularly the apparatus 110, in accordance with the disclosed invention.
  • Returning to FIG. 20 it should be noted that other than Web browser 220 on client 120, all the other components described in FIG. 2 may reside on the apparatus 110. However, a person skilled in the art would be able to adapt the disclosed architecture such that certain modification are made which provide for specific system 100 needs. For example, the database 118 may be distributed and accessible through the network 130. In another embodiment of the disclosed invention, a plurality of apparatus 110 handle the surveys and have access to a plurality of respective databases 118.
  • The code memory 116 contains, in addition-to other codes, a code that is targeted for the matching process required for the purpose of matching a user to a plurality of other users and establishing the user's unique affinity group. In contrast to prior art solutions, the invention disclosed herein provides for a dynamic creation of affinity groups, in addition to the comparison to previously stored affinity groups. The advantage of automatically detecting and creating affinity groups is in the fact the multiple parameters filled out by the user as part of the survey allow for increasingly accurately assessing the affinity group or groups with which a user should be associated with. A user of the system is therefore able to match his profile to that of others such as himself, and determine, for example, an expected salary for a certain job description or geographic area. However, the method is not limited for job related surveys and these are provided merely as an exemplary and no-limiting usage scenarios. A person skilled in the art would further note that the methods disclosed herein below may not necessarily require the exemplary apparatus disclosed above, and such methods may be applied to user profiles and affinity groups provided from other means, automated, manual, or combinations thereof.
  • In accordance with the disclosed invention, it is necessary to perform a series of matches between user profiles, for the purpose of providing customized reports. In one embodiment of the disclosed invention, a profile entered by a user is matched against a plurality of user profiles stored in the system. More specifically, the entered profile is matched to other profiles according to “matchgroups” (explained in more detail below) to determine if the profiles are similar, or like, each other. It is therefore possible to create a list of user profiles that are like the profile entered by the user, otherwise known as an affinity group. Using matchgroups, it is further possible to match a plurality of profiles against a goal, where a goal may be a set of parameters to which a subset of plurality of user profiles matches best according to a defined score and score threshold. It is further possible to match an entered profile to one or more affinity groups having known characteristics. In one embodiment of the invention such matching is performed at every stage of the answers provided by the user as the survey questions are presented and responded to. In another embodiment, such matching is applied after a completed survey, taking into account all attributes of the entered user profile, and producing a resulting group or groups of matching profiles according to the overall matching goal.
  • In accordance with the disclosed invention a matchgroup may be defined as a plurality of match fields and a plurality of affinity groups, each (match field, affinity group, and matchgroup) having a score value. If the entered profile has values matching other profiles' values according to the matchgroup, the matchgroup score is increased by the score value defined for the matching field. If the entered profile has values matching within an affinity group, the matchgroup score is increased by the affinity score value. The plurality of scores is tallied, allowing the determination of an overall score for the matchgroup. Furthermore, the process is repeated for any number of defined matchgroups having potentially different match fields and affinity groups, allowing the determination of an overall score for a defined goal. Matching profiles may be then included or excluded depending on threshold score values defined for this purpose, such as “Excellent Match,” “Good Match,” or “Bad Match.” The process of matching is hereinafter described in greater detail.
  • Reference is now made to FIG. 21, where an exemplary and non-limiting flowchart 300 for the purpose of creating a user specific affinity group is shown. In step S310, a user profile is received (hereinafter the “received user profile”). This may be a user profile that was prepared based on an automated survey system, discussed in more detail above. In step S320 a score is associated with each of the possible match groups as shown above. A match group has a plurality of fields, affinity groups, and respective weights or scores, which allows for combinations of attribute matches, may be considered more valuable than other combinations. For instance, it may be preferable to match a smaller subset of profiles for which a location is closer to the target profile. By employing matchgroups within the profile search goal document, sets of matches can also be ranked. This is useful, for instance, in ranking profiles higher where geographic proximity is desirable. By creating a set of affinities designed to relate profiles by regions, the profile search system using matchgroups targets profiles within local regions first, ranking them higher than profiles that meet other match criteria but are in outlying regions. For instance, the search goal may first only find profiles in the same or surrounding cities. By defining metropolitan region affinity groups, user profiles of people in the same metropolitan region can be found. However, there may not be enough valid data in the system to find user profiles in the same metropolitan region, and therefore, user profiles from a larger surrounding region, such as a state, or multi-state area, can be considered. By including affinities in a ranked fashion in matchgroups, user profiles that are closes” to compared to user profile may be located, such located user profiles having the highest score ranks. One skilled in the art would recognize that closeness could apply to any attribute of a profile in addition to location, such as experience level, job category, company type, age ranges, and others. In step S330, a search is performed matching a plurality of user profiles to that of the received user profile. As a result of the search, a list of scores is returned for each of the compared user profiles. In one embodiment of the invention, the list is returned ordered from the highest to the lowest score. A more detailed description of the search process is provided below. In step 8340, any number of score fitness ranges is assigned, for example, a score range defining an excellent fit, a score range defining a good fit, and a score range defining below acceptable fit. More specifically, if the score is from ‘0’ to ‘100’ then excellent fit may be scores of ‘90’ through ‘100,’ good fit having scores of ‘70’ through ‘90,’ and so on. In step S350, a report of the profiles best fitting the received user profile is provided, identifying those profiles that best fit as a new affinity group. In another embodiment of the invention, the created affinity group is checked against the already existing affinity groups and, if there is a match, no new affinity group is created. An advantage of the disclosed invention over prior art is in the creation of an affinity group, which includes at least the received user profile. This allows for a larger match scope than the static assignment of an affinity group commonly found in prior art solutions. A person skilled in the art may modify the disclosed flowchart to fit with the needs of specific programming languages, e.g. C++, or description languages, e.g. XML, without departing from the scope of the disclosed invention.
  • Referring to FIG. 22, an exemplary and non-limiting flowchart of step S330 of FIG. 3 for the profile search process is shown. The flow chart explains the search and matching process of step S330 discussed in FIG. 3 above. In step S330-10 a user profile to be matched to the user profile received in step S310 is selected (hereinafter the “selected user profile”). The selected user profile is one that was not previously selected during he specific execution of the current search and matchgroup. In step S330-20, the match group score of the selected user profile is determined in comparison with the received user profile. The match score may be a binary value, for example, ‘0’ meaning no match and ‘1’ meaning match. It is also possible to have a sliding degree of fitness with multiple fitness score values, for example, ‘0’=no match, ‘1’=bad match, ‘2’=acceptable match, ‘3’=good match, and ‘4’=excellent match. The specific match score is correlated further with the score value that is indicative of the weight a certain match field may have. As noted above, certain match fields may have a higher importance for the overall score than others, and hence have a higher score value, or effect, on the overall score result. It is further possible to limit the score, for example to a maximum score, for any matchgroup, so that a multitude of matches of a particular field do not skew the overall match score. In step S330-30, it is checked if more match fields within the same user profile are to be checked and if so execution continues with step S330-20. Otherwise, execution continues with step S330-40. In step S330-40, an overall fitness score respective of the user profile in comparison to the user profile received in step S210 is calculated. This is a weighted value that takes into consideration all the relevant matchgroups. The overall result is used as a score indicative of the level of fitness between the two user profiles. In one embodiment of this invention these scores are normalized, for example, ‘100%’ meaning a best fit and ‘0%’ being no fit.
  • A matchgroup may be classified as being deterministic or non-deterministic, and requiring at least one deterministic match to exist within the overall score, thus allowing a more deterministic match than solely a weighted fuzzy type of result. For example, a matchgroup designated for ‘Job Type’ could be marked as deterministic, and then no profile is included in the overall affinity group unless there is at least a score, which may also require to be at least above a predefined threshold value, given to the Job Type matchgroup for the profile. Consequently, any score in a non-deterministic matchgroup could serve to increase the overall score of a profile, but in this case cannot by itself be sufficient to allow the profile to be included in the final fit. One skilled in the art would recognize that parameters ‘such as this can be considered’ optional in different uses of the system, and are not meant to be limiting, but providing further flexibility.
  • In yet another embodiment of the disclosed invention it is further possible to score on ranges of values rather than by an exact match. For example, a matchgroup can define a match field for Age Range, having defined ranges of Less than 17, 18-29, 30-45, 45-64, and 65+. If the entered profile matches by range with any user profile to be compared, the user profile could receive a score for such a matchgroup. It is also possible to score based on other Boolean criteria, such as a match field existing in a profile and having any value at all, as opposed to not existing in a profile. It is further possible to limit matches by a fixed criteria, such as only to allow for matches with profiles constrained to a particular arbitrary date range.
  • In step S330-50, it is checked whether there are more user profiles not yet matched against and if so execution continues with step S330-10. Otherwise, execution continues with step S330-60. In step S330-60, a report including the score results of the user profiles to which the user profile received in step S310 was matched against are provided. These results, for example, may be used for the purpose of reporting by the apparatus 110 discussed in more detail above. The survey apparatus 110 uses this list to determine the affinity group to which a user belongs and to prepare tailored reports based, at least in part, on such an association to an affinity group or the determination of affinity groups as discussed in more detail above. A person skilled in the art may modify the disclosed flowchart to fit with the needs of specific programming languages, e.g. C++, or description languages, e.g. XML, without departing from the scope of the disclosed invention, or further be able to use it on an apparatus other than the one described as the apparatus 110.
  • Reference is now made to FIG. 23 where an exemplary and non-limiting extensible markup language (XML) code for profile matching is shown. The code may reside in memory code 116 of the apparatus 110. Lines 504 through 507 provide information of filters to be applied on the search data. This may include filters such as the state or a country in which a user is located or in which the user wishes to be. Another filter may be the profiles to be searched which can include all the profiles available, or a subset thereof. Lines 508 through 514 define sources for user profiles. Such sources may be available on database 118. Such sources may be affinity groups created by apparatus 110. Lines 516-522, 523-527, 528-532, 533-537, 538-541, 542-548, 549-552, 553-557, 558-561, 562-564, 565-567, 568-577, 578-581, 582-585, 586-590, and 591-594 are examples of matches performed in accordance with the disclosed invention, as made applicable to an XML implementation. Lines 595-598 define the ranking ranges so as to allow for the grouping of the matching profiles in accordance with the disclosed invention. A person skilled in the art may modify the disclosed code to adapt to other types of searches for the purpose of matching user profiles without departing from the scope of the disclosed invention.
  • Referring to FIG. 24 an exemplary and non-limiting flowchart 600 for the matching of a user profile to an affinity group, is shown. In step S610, an affinity group to be matched to a user profile, for example the user profile received in step S310 shown in FIG. 3, is selected. This affinity group is one that was not matched before during the specific execution of the current search and match. The match process entails the determination of scores of match groups of the affinity group with respect to the received profile. In step S620, a match group of the affinity group and the received user profile are compared to determine a score. The score may be a binary value, for example, ‘0’ meaning low and ‘1’ meaning high. It is also possible to have a sliding degree of fitness with multiple fitness score values, for example, ‘0’=no match, ‘1’=bad match, ‘2’=acceptable match, ‘3’=good match, and ‘4’=excellent match. The specific score may be further weighted based on the certain match group used. For example, the match group of location may have a higher weight than that of a job position. In step S630, it is checked if more match groups between the affinity group and the received user profile are to be checked and if so execution continues with step S620. Otherwise, execution continues with step S640. In step S640, an overall fitness score respective of the affinity group in comparison to the user profile is calculated. This is a weighted value that takes into consideration all the match fields compared and their respective score values. The overall result is used as a score, indicative of the level of fitness between the affinity group and the user profile. In one embodiment of this invention these scores are normalized, for example, ‘100%’ meaning a best fit and ‘0%’ being no fit. In step S650, it is checked whether there are more affinity groups not: yet matched against and if so execution continues with step S610. Otherwise, execution continues with step S660. In step S660, a report including the score results of the affinity groups to which the user profile was matched against are provided. A person skilled in the art would be able to extend the apparatus 110 to provide the necessary reports by, for example, including the required code in code memory 116. A person skilled in the art may further modify the disclosed flowchart to fit with the needs of specific programming languages, e.g. C++, or description languages, e.g. XML, without departing from the scope of the disclosed invention.
  • Reference is now made to FIG. 25, where an exemplary and non-limiting flowchart 700 for reporting of results of a matching process is shown. Such a report may be provided in response to a goal setting as described above, and in more detail in a U.S. patent application entitled AUTOMATED COMPENSATION REPORTS USING ONLINE SURVEYS AND COLLABORATIVE FILTERING, Serial Number (unknown), filed on the same day and date, assigned to common assignee, and which is hereby incorporated in its entirety by this reference thereto. In step S710, a user is asked if an extended report is desired and, if so, execution continues with step S730. Otherwise, execution continues with step S720, where a standard report is generated and, thereafter, execution continues with step S750. The report is generated based on the user profile and the respective affinity group associated with that user profile generated by the matching system. In step S730, billing information is received from the user, for example client 120-1. Such information may include, but is not limited to, a purchase order (PO) number, a credit card number, or a debit card number. In step S740, an extended report is generated, the report being based on the user profile and the respective affinity group associated to that user profile. An extended report may provide additional information otherwise not made available to the non-paying user. Specifically, such extended reports may include reports of a statistical nature respective of a plurality of users. This is the case where a human resource manager wants to find out certain statistics about compensation demands common in a certain geographical area. In step S750, the extended report or the standard report, as the case may be, is sent to the user, for example by means of displaying it as a Web page on the terminal of the client 120, as an e-mail attachment, and the like. In another embodiment of the invention the client 120 may further provides report parameters that influence the desired output of the report, for example, the type of information to be displayed, the manner in which such information is provided, and the like. Such information may be provided by means of defining goals as described in greater details in the above mentioned U.S. patent application.
  • Although the invention is described herein with reference to the preferred embodiment, one skilled in the art will readily appreciate that other applications may be substituted for those set forth herein without departing from the spirit and scope of the present invention. Accordingly, the invention should only be limited by the Claims included below.

Claims (103)

1. A method for surveying a user, comprising the steps of:
presenting at least one question to said user, said at least one question being pertinent for determination of at least one affinity group to said user;
creating a user profile for said user based on said user's answers to said at least one question, wherein said user profile is comprised of said at least one question and at least a corresponding answer;
adding said user profile to one or more affinity groups; and
storing said user profile.
2. The method of claim 1, wherein said question is comprised of a set of one or more question fields that form a logical grouping of questions into a single record.
3. The method of claim 1, wherein said method further comprises the step of:
presenting at least one additional question which is a result of said at least a question presented to said user.
4. The method of claim 3, wherein said at least another question is presented as a result of at least one of:
popularity of said at least another question within said at least an affinity group;
being a most frequently answered said at least another question within said at least an affinity group;
being a most recently answered said at least another question within said at least an affinity group; and
relation to said at least a question.
5. The method of claim 1, wherein said at least a corresponding answer is an open text.
6. The method of claim 5, wherein said open text allows said user to add a new answer to said at least a corresponding answer.
7. The method of claim 6, wherein said new answer is used as one of said respective answers when said at least as question is suggested.
8. The method of claim 1, wherein said at least a question may have one or more possible answers displayed.
9. The method of claim 1, wherein said user may select one or more answers to said question.
10. The method of claim 8, wherein said one or more possible answers displayed is based on at least one of:
popularity of an answer within said at least an affinity group;
being a most frequent answer within said at least an affinity group; and being a most recent answer within said at least an affinity group.
11. The method of claim 1, further comprising the step of:
filtering said user's answers to said question.
12. The method of claim 11, wherein said filtering is performed in at least one of real-time and batch mode.
13. The method of claim 11, wherein said filtering further comprises the modification of an answer to said question based on at least one of:
predetermined criterion;
one or more filtering rules;
consistency with previously answered questions;
consistency with all answered questions; and
consistency with answers of at least said affinity group.
14. The the method of claim 1, further comprising the steps of:
periodically creating a new affinity group; and
associating at least a user to said new affinity group.
15. The method of claim 1, further comprising the step of:
creating a new affinity group by a user.
16. The method of claim 15, further comprising the step of:
associating matching user profiles with said new affinity group.
17. The method of claim 1, further comprising:
the steps of providing a report.
18. The method of claim 17, wherein the generation of said report is constrained by at least one of an attribute of said user profile and a goal.
19. The method of claim 18, wherein said goal comprises at least one of:
a profile attribute value;
a range of profile attribute values;
a Boolean value reflective of a set of values;
a Boolean value reflective of a set of non-values;
a date range;
a minimum count of matching profiles;
a maximum count of matching profiles;
a question filter; and
a match score threshold.
20. The method of claim 18, wherein said constraint results are weighted such that deterministic fields have a higher score then non-deterministic fields.
21. The method of claim 18, wherein for the purpose of providing said report, said method comprises the step of matching to at least one of said affinity groups at least one of a goal and a user profile.
22. The method of claim 21, further comprising the step of:
determining the best match.
23. The method of claim 21, further comprising the step of:
determining one or more matches which are the best matches from a plurality of possible matches.
24. The method of claim 23, wherein said matches are arranged by order of match score, said order being grouped to at least two levels of matches, based on said score.
25. The method of claim 21, wherein said step of matching further comprises the steps of:
scoring of each match with said affinity groups; and
weighting the importance of including each said match in the aggregation of said report.
26. The method of claim 21, wherein said step of matching to at least one of said affinity groups further comprises the steps of:
classifying of each match with said affinity groups as being one of key in meeting said goal and ancillary in meeting said goal.
27. The method of claim 18, wherein said goal is one of a personal goal and an informative goal.
28. The method of claim 27, wherein said informative goal provides aggregated information pertaining to a plurality of user profiles.
29. The method of claim 28, wherein said aggregated information comprises at least statistical information.
30. The method of claim 27, wherein said report resulting of said personal goal provides a comparison of the user having said user profile to at least one of a plurality of user profiles and an affinity group.
31. A method for a compensation survey of a user, comprising the steps of:
presenting at least one question to said user, said at least one question being pertinent for determination of at least compensation within an affinity group of said user;
creating a user profile for said user based on said user's answers to said at least one question, wherein said user profile is comprised of said at least one question and at least one corresponding answer;
adding said user profile to one or more affinity groups; and
storing said user profile.
32. The method of claim 31, wherein said question is comprised of a set of one or more question fields that form a logical grouping of questions into a single record.
33. The method of claim 31, wherein said method further comprises the step of presenting at least another question which is a result of said at least a question presented to said user.
34. The method of claim 33, wherein said at least another question is presented as a result of at least one of: popularity of said at least another question within said at least an affinity group, being a most frequently answered said at least another question within said at least an affinity group, being a most recently answered said at least another question within said at least an affinity group, relation to said at least a question.
35. The method of claim 31, wherein said at least a corresponding answer is an open text.
36. The method of claim 35, wherein said open text allows said user to add a new answer to said at least a corresponding answer.
37. The method of claim 36, wherein said new answer is used as one of said respective answers when said at least as question is suggested.
38. The method of claim 31, wherein said at least a question may have one or more possible answers displayed.
39. The method of claim 31, wherein said user may select one or more answers to said question.
40. The method of claim 38, wherein said one or more possible answers displayed is based on any of:
popularity of an answer within said at least an affinity group;
being a most frequent answer within said at least an affinity group; and
being a most recent answer within said at least an affinity group.
41. The method of claim 31, further comprising the step of:
filtering said user's answers to said question.
42. The method of claim 41, wherein said filtering is performed in at least one of real-time and batch mode.
43. The method of claim 41, said filtering further comprising the step of:
modifying an answer to said question based on any of:
predetermined criterion;
one or more filtering rules;
consistency with previously answered questions;
consistency with all answered questions; and
consistency with answers of at least said affinity group.
44. The method of claim 31, further comprising the steps of:
periodically creating a new affinity group; and
associating at least one user to said new affinity group.
45. The method of claim 31, further comprising the step of:
creating a new affinity group by a user.
46. The method of claim 45, further comprising the step of:
associating matching user profiles with said new affinity group.
47. The method of claim 31, wherein said affinity group comprises any of:
profession;
geographic location;
compensation;
compensation range;
experience;
experience range;
position; and
position range.
48. The method of claim 31, further comprising the step of:
providing a compensation report.
49. The method of claim 48, wherein the generation of said compensation report is constrained by at least one of an attribute of said user profile and a goal.
50. The method of claim 49, further comprising the step of:
weighting said constraint results, wherein deterministic fields have a higher score than non-deterministic fields.
51. The method of claim 49, wherein said goal comprises any of:
a desired compensation;
a desired range of compensation;
a desired geographic location;
a desired firm; and
a desired range of firms.
52. The method of claim 49, wherein for the purpose of providing said report, said method comprises the step of:
matching to at least one of said affinity groups at least one of a goal and a user profile.
53. The method of claim 52, further comprising the step of:
determining a best match.
54. The method of claim 52, further comprising the step of:
determining one or more matches which are best matches from a plurality of possible matches.
55. The method of claim 54, wherein said matches are arranged by order of match score, said order being grouped to at least two levels of matches, based on said score.
56. The method of claim 52, wherein said step of matching further comprises the steps of:
the scoring of each match with said affinity groups; and
weighting importance of including best match in an aggregation of said report.
57. The method of claim 49, wherein said goal is any of a personal goal and an informative goal.
58. The method of claim 57, wherein said personal goal is any of:
compensation;
compensation range;
salary;
salary range;
geographic location;
position; and
position range.
59. The method of claim 57, wherein said informative goal provides aggregated information pertaining to a plurality of user profiles.
60. The method of claim 59, wherein said aggregated information comprises at least statistical information.
61. The method of claim 60, wherein said statistical information comprises any of:
average compensation;
average salary;
median compensation;
median salary;
salary percentile;
standard deviation;
trend;
profile count;
most frequent compensation; and
most frequent salary.
62. The method of claim 57, wherein said report resulting from said personal goal provides a comparison of a user having said user profile to at least one of a plurality of user profiles and an affinity group.
63. The method of claim 31, wherein said compensation any of:
annual salary;
monthly salary;
weekly salary;
hourly rate;
bonus;
tip;
benefits; and
vacation time.
64. A system for compensation surveying and reporting, comprising:
means for accessing a Website, said Website being connected to a network;
means for surveying a user for information pertinent for determination of compensation;
means for creating a user profile for said user based on gathering of said information, said user profile further comprised of at least one question and one corresponding answer;
means for storing said user profile; and
means for matching an attribute of said user profile with at least affinity group, said affinity groups being previously stored in said storage means.
65. A method for matching a user profile to a plurality of user profiles, comprising the steps of:
receiving a first user profile;
assigning a score value to at least one match group;
matching said first user profile with at least a second user profile from said plurality of user profiles and assigning an overall fitness score; and
reporting said fitness score.
66. The method of claim 65, wherein said score value is a relative weight of said match group.
67. The method of claim 66, wherein said weight is proportionate to at least the importance of a score value of said match group to the said overall fitness score.
68. The method of claim 66, wherein said match group is further comprised of at least one of match field and an affinity group.
69. The method of claim 68, wherein said match field comprises a score value.
70. The method of claim 68, wherein said affinity group comprises a score value.
71. The method of claim 65, wherein said match group comprises further one of a deterministic and a non-deterministic match group.
72. The method of claim 71, wherein a score is determined for a profile only if there exists a match to at least said deterministic match group.
73. The method of claim 65, wherein said matching comprises the steps of:
a) selecting said second user profile from a plurality of user profiles not previously matched with said first user profile;
b) comparing a match field of said first user profile to a respective match field of said second user profile, and determining a match group score;
c) repeating step b) until all match fields requiring a comparison are matched between said first user profile and said second user profile;
d) assigning a fitness score to said second user profile based on the plurality of said scores determined in step b);
e) repeating the method steps from step a) until all available user profiles are matched against said first user profile;
f) reporting the fitness scores.
74. The method of claim 73, wherein said step b) further comprises the step of matching at least one of a match field and an affinity.
75. The method of claim 74, wherein said fitness score is at least a tally of score values of at least one of a match group, a match field, and an affinity.
76. The method of claim 73, wherein said fitness scores of step f) are sorted by order of fitness.
77. The method of claim 76, wherein user profiles having a highest fitness scores are grouped into an affinity group.
78. The method of claim 77, wherein a number of user profiles is determined using any of:
a minimum fitness score of a user profile;
range of fitness scores;
an absolute number of the user profiles;
a defined percent of user profiles; and
a date range of user profiles.
79. The method of claim 65, further comprising the steps of:
receiving information of a type of report requested;
if said report is of a type requiring receipt of financial commitment, then receiving relevant information;
providing the desired report.
80. The method of claim 79, wherein said receipt of financial commitment comprises the step of
receiving any of credit card information, debit card information, purchase order information, coupon information, and check information.
81. The method of claim 65, wherein said plurality of user profiles comprises an affinity group.
82. The method of claim 65, wherein said fitness score is at least a weighted score of at least one of a match group, a match field, and an affinity group.
83. A computer software product containing instructions for matching a user profile to a plurality of user profiles, the instructions further being capable of executing at least the steps of:
receiving a first user profile;
assigning a score value to at least a match group;
matching said first user profile with at least a second user profile from said plurality of user profiles;
assigning a fitness score to said at least second user profile; and
reporting the fitness score of at least said second user profile.
84. The computer software product of claim 83, wherein said matching step comprises the steps of:
a) selecting said second user profile from a plurality of user profiles not previously matched with said first user profile;
b) comparing a match field of said first user profile to a respective match field of said second user profile, and determining a score;
c) repeating step b) until ail match fields requiring a comparison are matched between said first user profile and said second user profile;
d) assigning a fitness score to said second user profile based on the plurality of said scores determined in step b);
e) repeating the method from step a) until all available user profiles are matched against said first user profile; and
f) reporting the fitness scores.
85. The computer software product of claim 84, wherein said step b) further comprises the step of:
matching at least one of a match field and an affinity.
86. The computer software product of claim 83, further comprising the steps of:
receiving information of the type of report requested;
if said report is of a type requiring receipt of financial commitment, then receiving relevant information; and
providing a desired report.
87. A method for matching a user profile to at least an affinity group, comprising the steps of:
receiving a first user profile;
assigning a score value to at least a match group;
matching said first user profile with said at least an affinity of said affinity group;
assigning a fitness score to said affinity group; and
reporting said fitness score.
88. The method of claim 87, wherein said matching step comprises the steps of:
a) selecting said first affinity group from a plurality of affinity groups not previously matched with said first user profile;
b) comparing a match field of said first user profile to a respective match field of said first affinity, and determining a match group score;
c) repeating step b) until all match groups requiring a comparison are matched between said first user profile and said first affinity;
d) assigning a fitness score to said affinity group based on the plurality of said scores determined in step b);
e) repeating the method from step a) until all available affinity groups are matched against said first user profile; and
f) reporting the fitness scores.
89. The method of claim 87, further comprising the steps of:
receiving information of a type of report requested;
if said report is of a type requiring receipt of financial commitment, then receiving the relevant information; and
providing a desired report.
90. A computer software product containing instructions for matching a user profile to at least an affinity group, the instructions further being capable of executing at least the steps of:
receiving a first user profile;
assigning a score value to at least a match group;
matching said first user profile with at least an affinity of said affinity;
assigning a fitness score to said affinity group; and
reporting the fitness score of at least said first affinity group.
91. The computer software product of claim 90, wherein said matching step comprises the steps of:
a) selecting said first affinity group from a plurality of affinity groups not previously matched with said first user profile;
b) comparing a match field of said first user profile to the respective match field of said first affinity, and determining a match group score;
c) repeating step b) until all match groups requiring a comparison are matched between said first user profile and said first affinity;
d) assigning a fitness score to said affinity group based on a plurality of said scores determined in step b);
e) repeating the method from step a) until all available affinity groups are matched against said first user profile; and
f) reporting fitness scores.
92. The computer software product of claim 90, further comprising the steps of:
receiving information of a type of report requested;
if said report is of a type requiring receipt of financial commitment, then receiving relevant information; and
providing a desired report.
93. An apparatus for performing automated surveys, comprising:
database means containing at least a plurality of prospective survey questions and prospective answers;
memory means for storing code, wherein said code contains at least code to perform an automated survey using said plurality of survey questions and prospective answers, for selecting a questions and respective prospect answers based on at least a response provided to a previous question, code for creating a user profile, code for creating at least an affinity group based on a plurality of user profiles, and code for association of said user profile with at least an affinity group;
computational means for executing said code; and
communication means for communication with at least a user.
94. The apparatus of claim 93, said database means further comprising:
at least one of user profiles database, affinity groups database, and reporting definitions.
95. The apparatus of claim 93, wherein said code to perform an automated survey selects one of said plurality of survey questions based on at least one of answers provided by said user, said user profile, and an affinity group.
96. The apparatus of claim 93, further comprising:
means for periodically executing said code for creating at least an affinity group based on a plurality of user profiles.
97. The apparatus of claim 93, wherein said apparatus further comprises:
means for providing at least a report.
98. The apparatus of claim 97, further comprising
means for providing said at least a report as at least one of a basic report and an extended report.
99. The apparatus of claim 98, further comprising
means for allowing free access to said basic report.
100. The apparatus of claim 98, further comprising:
means for receiving billing information for providing said extended report.
101. The apparatus of claim 97, wherein said report is provided in view of at least one of a user profile and an affinity group.
102. The apparatus of claim 93, wherein said communication means comprises at least one of a local area network, a wide area network, and the world-wide-web.
103. A system for the purpose of performing automated surveys, comprising:
a network connected to at least a client, said client configured with at least a Web browser;
a database for storing at least a plurality of survey questions and prospective answers; and,
a server having access to said network and said database, said server configured to at least survey at least a user using said plurality of survey questions and prospective answers and for selecting said questions and respective prospect answers based on at least a response provided to a previous question, for creating user profiles, for creating at least an affinity group based on a plurality of user profiles, and for associating said user profile with at least an affinity group.
US10/541,156 2002-12-27 2003-12-29 Surveying apparatus and method for compensation reports Abandoned US20060111959A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US10/541,156 US20060111959A1 (en) 2002-12-27 2003-12-29 Surveying apparatus and method for compensation reports

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US43680902P 2002-12-27 2002-12-27
US10/541,156 US20060111959A1 (en) 2002-12-27 2003-12-29 Surveying apparatus and method for compensation reports
PCT/US2003/041413 WO2004061740A1 (en) 2002-12-27 2003-12-29 A surveying apparatus and method for compensation reports

Publications (1)

Publication Number Publication Date
US20060111959A1 true US20060111959A1 (en) 2006-05-25

Family

ID=32713093

Family Applications (3)

Application Number Title Priority Date Filing Date
US10/748,968 Expired - Fee Related US7571110B2 (en) 2002-12-27 2003-12-29 Automated compensation reports using online surveys and collaborative filtering
US10/541,156 Abandoned US20060111959A1 (en) 2002-12-27 2003-12-29 Surveying apparatus and method for compensation reports
US12/496,916 Abandoned US20090265224A1 (en) 2002-12-27 2009-07-02 Generation of Automated Reports and Matching Using Online Surveys and Collaborative Filtering

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US10/748,968 Expired - Fee Related US7571110B2 (en) 2002-12-27 2003-12-29 Automated compensation reports using online surveys and collaborative filtering

Family Applications After (1)

Application Number Title Priority Date Filing Date
US12/496,916 Abandoned US20090265224A1 (en) 2002-12-27 2009-07-02 Generation of Automated Reports and Matching Using Online Surveys and Collaborative Filtering

Country Status (5)

Country Link
US (3) US7571110B2 (en)
EP (1) EP1579364A4 (en)
CN (1) CN1754181A (en)
AU (1) AU2003300404A1 (en)
WO (1) WO2004061740A1 (en)

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040230989A1 (en) * 2003-05-16 2004-11-18 Macey William H. Method and apparatus for survey processing
US20050268217A1 (en) * 2004-05-27 2005-12-01 Collegenet, Inc. System for describing the overlaying of electronic data onto an electronic image
US20070266054A1 (en) * 2006-03-24 2007-11-15 Stephens Frank M Method and system for salary planning and performance management
US20080270218A1 (en) * 2004-05-11 2008-10-30 You Know ? Pty Ltd System and Method for Obtaining Pertinent Real-Time Survey Evidence
US20090077061A1 (en) * 2007-09-19 2009-03-19 Abercrombie Iii Charles Clinton Social network for travelers with layovers
US7650286B1 (en) * 2003-04-18 2010-01-19 Algomod Technologies Corporation Recruitment vendor management system and method
US20100082361A1 (en) * 2008-09-29 2010-04-01 Ingenix, Inc. Apparatus, System and Method for Predicting Attitudinal Segments
US7979291B2 (en) * 2005-03-22 2011-07-12 Ticketmaster Computer-implemented systems and methods for resource allocation
US20110191106A1 (en) * 2005-06-24 2011-08-04 American Express Travel Related Services Company, Inc. Word recognition system and method for customer and employee assessment
US8103659B1 (en) * 2005-06-06 2012-01-24 A9.Com, Inc. Perspective-based item navigation
US8122371B1 (en) * 2007-12-21 2012-02-21 Amazon Technologies, Inc. Criteria-based structured ratings
US8401884B1 (en) * 2005-11-07 2013-03-19 Avantas L.L.C. Electronic scheduling for work shifts
US8517742B1 (en) * 2005-05-17 2013-08-27 American Express Travel Related Services Company, Inc. Labor resource testing system and method
US8676615B2 (en) 2010-06-15 2014-03-18 Ticketmaster Llc Methods and systems for computer aided event and venue setup and modeling and interactive maps
US20150058240A1 (en) * 2007-04-30 2015-02-26 Experteer Gmbh Method and System for Providing a Virtual Job Market
US8977654B1 (en) * 2012-09-21 2015-03-10 Google Inc. Assigning classes to users of an online community
US9325755B1 (en) * 2007-09-19 2016-04-26 Intercept, Llc Social network for travelers with layovers
US9608929B2 (en) 2005-03-22 2017-03-28 Live Nation Entertainment, Inc. System and method for dynamic queue management using queue protocols
US9781170B2 (en) 2010-06-15 2017-10-03 Live Nation Entertainment, Inc. Establishing communication links using routing protocols
US9912653B2 (en) 2007-09-04 2018-03-06 Live Nation Entertainment, Inc. Controlled token distribution to protect against malicious data and resource access
US10573084B2 (en) 2010-06-15 2020-02-25 Live Nation Entertainment, Inc. Generating augmented reality images using sensor and location data
US20210374769A1 (en) * 2008-03-05 2021-12-02 Ebay Inc. Method and apparatus for social network qualification systems

Families Citing this family (65)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9819561B2 (en) 2000-10-26 2017-11-14 Liveperson, Inc. System and methods for facilitating object assignments
US8868448B2 (en) 2000-10-26 2014-10-21 Liveperson, Inc. Systems and methods to facilitate selling of products and services
US20040236598A1 (en) * 2003-03-21 2004-11-25 Thomsen David J. System and method for providing occupational information
US20110076663A1 (en) * 2003-08-18 2011-03-31 Retail Optimization International Systems and methods for selecting survey questions and available responses
US9152624B1 (en) 2003-12-04 2015-10-06 Retail Optimization International, Inc. Systems and methods for visual presentation and navigation of content using data-based image analysis
US7877679B2 (en) * 2005-05-04 2011-01-25 Amadesa Ltd. System and method for generating a user profile from layers based on prior user response
US9432468B2 (en) 2005-09-14 2016-08-30 Liveperson, Inc. System and method for design and dynamic generation of a web page
US8738732B2 (en) 2005-09-14 2014-05-27 Liveperson, Inc. System and method for performing follow up based on user interactions
US8781899B2 (en) * 2005-11-28 2014-07-15 Voiceport, Llc Advertising a pharmaceutical product to a third party
US20070192161A1 (en) * 2005-12-28 2007-08-16 International Business Machines Corporation On-demand customer satisfaction measurement
US10210530B1 (en) * 2006-08-11 2019-02-19 Infor (Us), Inc. Selecting a report
US20080120323A1 (en) * 2006-11-17 2008-05-22 Lehman Brothers Inc. System and method for generating customized reports
US8175990B1 (en) 2007-01-04 2012-05-08 Iloop Mobile, Inc. Situational decision engine and method for contextual user experience
US9135627B2 (en) * 2007-07-11 2015-09-15 Dynamic Logic, Inc. Method and system for on-line survey recruitment
US20090089269A1 (en) * 2007-10-02 2009-04-02 Jacqueline Linder Method and system for managing information in a database
US20090222305A1 (en) * 2008-03-03 2009-09-03 Berg Jr Charles John Shopper Communication with Scaled Emotional State
AU2008100458B4 (en) * 2008-04-30 2008-10-02 Anthony Sork Attachment measurement device, system and methodology
US8577884B2 (en) * 2008-05-13 2013-11-05 The Boeing Company Automated analysis and summarization of comments in survey response data
US8762313B2 (en) 2008-07-25 2014-06-24 Liveperson, Inc. Method and system for creating a predictive model for targeting web-page to a surfer
US8260846B2 (en) 2008-07-25 2012-09-04 Liveperson, Inc. Method and system for providing targeted content to a surfer
US8805844B2 (en) 2008-08-04 2014-08-12 Liveperson, Inc. Expert search
US9892417B2 (en) 2008-10-29 2018-02-13 Liveperson, Inc. System and method for applying tracing tools for network locations
US9147177B2 (en) * 2008-11-07 2015-09-29 Oracle International Corporation Method and system for implementing a scoring mechanism
US20100121685A1 (en) * 2008-11-07 2010-05-13 Oracle International Corporation Method and System for Implementing a Ranking Mechanism
US9032311B2 (en) * 2008-11-07 2015-05-12 Oracle International Corporation Method and system for implementing a compensation system
US10275530B2 (en) * 2009-02-02 2019-04-30 Excalibur Ip, Llc System and method for communal search
US8166104B2 (en) * 2009-03-19 2012-04-24 Microsoft Corporation Client-centered usage classification
US8572227B2 (en) * 2009-03-27 2013-10-29 Bank Of America Corporation Methods and apparatuses for communicating preservation notices and surveys
US20100257035A1 (en) * 2009-04-07 2010-10-07 Microsoft Corporation Embedded content brokering and advertisement selection delegation
US20100262547A1 (en) * 2009-04-14 2010-10-14 Microsoft Corporation User information brokering
US20100306024A1 (en) * 2009-05-29 2010-12-02 Vision Critical Communications Inc. System and method of providing an online survey and summarizing survey response data
US20110054972A1 (en) * 2009-08-28 2011-03-03 Oracle International Corporation Calculation/determination of budget and employee allocation targets using a model
US9129263B2 (en) * 2009-12-01 2015-09-08 Yahoo! Inc. System and method for automatically building up topic-specific messaging identities
EP2556449A1 (en) 2010-04-07 2013-02-13 Liveperson Inc. System and method for dynamically enabling customized web content and applications
US8918465B2 (en) 2010-12-14 2014-12-23 Liveperson, Inc. Authentication of service requests initiated from a social networking site
US9350598B2 (en) 2010-12-14 2016-05-24 Liveperson, Inc. Authentication of service requests using a communications initiation feature
US8909587B2 (en) * 2011-11-18 2014-12-09 Toluna Usa, Inc. Survey feasibility estimator
US8943002B2 (en) 2012-02-10 2015-01-27 Liveperson, Inc. Analytics driven engagement
US8805941B2 (en) 2012-03-06 2014-08-12 Liveperson, Inc. Occasionally-connected computing interface
US9563336B2 (en) 2012-04-26 2017-02-07 Liveperson, Inc. Dynamic user interface customization
US9672196B2 (en) 2012-05-15 2017-06-06 Liveperson, Inc. Methods and systems for presenting specialized content using campaign metrics
US20140032382A1 (en) * 2012-07-30 2014-01-30 Platypus Business Systems, Inc. Compensation recommendations
US10515380B2 (en) * 2012-11-16 2019-12-24 [24]7.ai, Inc. Proactive surveys based on customer information
US8805699B1 (en) 2012-12-21 2014-08-12 Reputation.Com, Inc. Reputation report with score
US8744866B1 (en) 2012-12-21 2014-06-03 Reputation.Com, Inc. Reputation report with recommendation
US20140229236A1 (en) * 2013-02-12 2014-08-14 Unify Square, Inc. User Survey Service for Unified Communications
WO2015017380A1 (en) * 2013-07-29 2015-02-05 SR Solutions, Inc. Gathering user information based on user interactions
US9779177B1 (en) * 2014-01-31 2017-10-03 EMC IP Holding Company LLC Service generation based on profiled data objects
US11386442B2 (en) 2014-03-31 2022-07-12 Liveperson, Inc. Online behavioral predictor
US9697498B2 (en) * 2014-07-23 2017-07-04 Linkedin Corporation Inferred salary distribution for schools
US10198738B2 (en) 2014-08-25 2019-02-05 Accenture Global Services Limited System architecture for customer genome construction and analysis
US10142908B2 (en) 2015-06-02 2018-11-27 Liveperson, Inc. Dynamic communication routing based on consistency weighting and routing rules
US10515331B1 (en) 2016-05-27 2019-12-24 Vega Factor Inc. System and method for evaluating individuals and modeling compensation levels
US10721509B2 (en) 2016-07-27 2020-07-21 Accenture Global Solutions Limited Complex system architecture for sensatory data based decision-predictive profile construction and analysis
CN116528220A (en) 2016-08-14 2023-08-01 利维帕尔森有限公司 Method and system for real-time remote control of mobile applications
CN106530077A (en) * 2016-11-10 2017-03-22 四川享宇金信金融服务外包有限公司 Intelligent information survey device and method
US20220300306A1 (en) * 2017-05-16 2022-09-22 BlueOwl, LLC Artificial intelligence based computing systems and methods for providing enhanced user help
CN107578183B (en) * 2017-09-18 2020-04-21 上海量明科技发展有限公司 Resource management method and device based on capability evaluation
US10592647B2 (en) * 2017-09-25 2020-03-17 International Business Machines Corporation Authentication using cognitive analysis
US10949399B2 (en) * 2018-04-16 2021-03-16 Qualtrics, Llc Dynamically synchronizing electronic surveys and collaborative survey representation documents
US10770069B2 (en) * 2018-06-07 2020-09-08 International Business Machines Corporation Speech processing and context-based language prompting
US11645625B2 (en) 2018-08-21 2023-05-09 Job Market Maker, Llc Machine learning systems for predictive targeting and engagement
US11734701B2 (en) * 2019-09-11 2023-08-22 International Business Machines Corporation Cognitive dynamic goal survey
WO2021067939A1 (en) 2019-10-04 2021-04-08 Job Market Maker, Llc Machine learning systems and methods for predictive engagement
US20220327460A1 (en) * 2021-04-13 2022-10-13 At&T Intellectual Property I, L.P. Systems and methods for scaled engineering

Citations (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5197004A (en) * 1989-05-08 1993-03-23 Resumix, Inc. Method and apparatus for automatic categorization of applicants from resumes
US5339239A (en) * 1989-10-13 1994-08-16 Mitsubishi Plastics Industries Limited Information collecting and/or service furnishing systems by which a user can request information from a central data base using a portable personal terminal and an access terminal
US5416694A (en) * 1994-02-28 1995-05-16 Hughes Training, Inc. Computer-based data integration and management process for workforce planning and occupational readjustment
US5446919A (en) * 1990-02-20 1995-08-29 Wilkins; Jeff K. Communication system and method with demographically or psychographically defined audiences
US5659731A (en) * 1995-06-19 1997-08-19 Dun & Bradstreet, Inc. Method for rating a match for a given entity found in a list of entities
US5717923A (en) * 1994-11-03 1998-02-10 Intel Corporation Method and apparatus for dynamically customizing electronic information to individual end users
US5754939A (en) * 1994-11-29 1998-05-19 Herz; Frederick S. M. System for generation of user profiles for a system for customized electronic identification of desirable objects
US5761662A (en) * 1994-12-20 1998-06-02 Sun Microsystems, Inc. Personalized information retrieval using user-defined profile
US5832497A (en) * 1995-08-10 1998-11-03 Tmp Worldwide Inc. Electronic automated information exchange and management system
US5950200A (en) * 1997-01-24 1999-09-07 Gil S. Sudai Method and apparatus for detection of reciprocal interests or feelings and subsequent notification
US5948040A (en) * 1994-06-24 1999-09-07 Delorme Publishing Co. Travel reservation information and planning system
US5978768A (en) * 1997-05-08 1999-11-02 Mcgovern; Robert J. Computerized job search system and method for posting and searching job openings via a computer network
US6089284A (en) * 1998-09-24 2000-07-18 Marconi Commerce Systems Inc. Preconditioning a fuel dispensing system using a transponder
US6213780B1 (en) * 1998-07-06 2001-04-10 Chi Fai Ho Computer-aided learning and counseling methods and apparatus for a job
US6266659B1 (en) * 1997-08-07 2001-07-24 Uday P. Nadkarni Skills database management system and method
US6289340B1 (en) * 1999-08-03 2001-09-11 Ixmatch, Inc. Consultant matching system and method for selecting candidates from a candidate pool by adjusting skill values
US6385620B1 (en) * 1999-08-16 2002-05-07 Psisearch,Llc System and method for the management of candidate recruiting information
US6406302B1 (en) * 2000-02-14 2002-06-18 Webb Nelson System and method for collecting and exchanging survey data
US20020133502A1 (en) * 2001-01-05 2002-09-19 Rosenthal Richard Nelson Method and system for interactive collection of information
US6587832B1 (en) * 2000-03-16 2003-07-01 Compensate.Com Llc Market pay system
US20030145015A1 (en) * 2002-01-25 2003-07-31 Theodore Turnasella System and method providing user definable on-line wage and salary reports
US6618734B1 (en) * 2000-07-20 2003-09-09 Spherion Assessment, Inc. Pre-employment screening and assessment interview process

Family Cites Families (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4748A (en) * 1846-09-10 adams
US145015A (en) * 1873-11-25 Improvement in railway-rail fastenings
US2479A (en) * 1842-03-04 Machine for sizing papek
US204439A (en) * 1878-06-04 Improvement in chisels
US187725A (en) * 1877-02-27 Improvement in molding glass
US6443840B2 (en) * 1986-03-10 2002-09-03 Response Reward Systems, L.C. Evaluation of responses of participatory broadcast audience with prediction of winning contestants; monitoring, checking and controlling of wagering, and automatic crediting and couponing
US5259766A (en) * 1991-12-13 1993-11-09 Educational Testing Service Method and system for interactive computer science testing, anaylsis and feedback
WO1997026729A2 (en) * 1995-12-27 1997-07-24 Robinson Gary B Automated collaborative filtering in world wide web advertising
US5704017A (en) * 1996-02-16 1997-12-30 Microsoft Corporation Collaborative filtering utilizing a belief network
US5867799A (en) * 1996-04-04 1999-02-02 Lang; Andrew K. Information system and method for filtering a massive flow of information entities to meet user information classification needs
US20020002482A1 (en) * 1996-07-03 2002-01-03 C. Douglas Thomas Method and apparatus for performing surveys electronically over a network
US6477504B1 (en) * 1998-03-02 2002-11-05 Ix, Inc. Method and apparatus for automating the conduct of surveys over a network system
US6064980A (en) * 1998-03-17 2000-05-16 Amazon.Com, Inc. System and methods for collaborative recommendations
US6389400B1 (en) * 1998-08-20 2002-05-14 Sbc Technology Resources, Inc. System and methods for intelligent routing of customer requests using customer and agent models
US6519571B1 (en) * 1999-05-27 2003-02-11 Accenture Llp Dynamic customer profile management
IL133617A0 (en) * 1999-12-20 2001-04-30 Glide Ltd Career management system
US7188076B2 (en) * 1999-12-20 2007-03-06 Ndex Systems Inc. System and method for creating a true customer profile
US20010032097A1 (en) * 2000-04-28 2001-10-18 Levey Jonah S. Method and system for collecting and presenting information relating to compensations
US6862596B2 (en) * 2000-05-09 2005-03-01 Eri Economic Research Institute, Inc. System and method for retrieving and displaying data, such as economic data relating to salaries, cost of living and employee benefits
US6701322B1 (en) * 2000-06-07 2004-03-02 Ge Financial Assurance Holdings, Inc. Interactive customer-business interview system and process for managing interview flow
AU2001281017A1 (en) * 2000-08-03 2002-02-18 Unicru, Inc. Electronic employee selection systems and methods
US6741993B1 (en) * 2000-08-29 2004-05-25 Towers Perrin Forster & Crosby, Inc. Competitive rewards benchmarking system and method
US7711580B1 (en) * 2000-10-31 2010-05-04 Emergingmed.Com System and method for matching patients with clinical trials
US6626679B2 (en) * 2000-11-08 2003-09-30 Acesync, Inc. Reflective analysis system
US7376569B2 (en) * 2001-03-30 2008-05-20 Salary.Com Apparatus and method for providing compensation information
US20020188542A1 (en) * 2001-04-13 2002-12-12 Yong Zhang Compensation-data processing
US20020188489A1 (en) * 2001-05-22 2002-12-12 International Business Machines Corporation System and method for optimizing office worker productivity
US7805354B2 (en) * 2001-06-15 2010-09-28 Salary.Com Compensation data prediction
US20030046140A1 (en) * 2001-09-05 2003-03-06 Professor Mac, Llc Methods and systems for delivering market research services via a network
US7269579B2 (en) * 2001-09-18 2007-09-11 Lovegren Victoria M Method for tracking and assessing program participation
US20030083921A1 (en) * 2001-10-29 2003-05-01 Clark La Verne Methods and systems for exchanging information, such as information related to recruiting foreign personnel for domestic-based companies
US7054864B1 (en) * 2001-11-02 2006-05-30 Paul Toomey Labor market information analyzer system for researchers, employers, staff, and others
US6775518B2 (en) * 2002-01-25 2004-08-10 Svi Systems, Inc. Interactive education system
US20030187725A1 (en) 2002-03-29 2003-10-02 Jotkowitz Joseph Brent Monitoring professional development
US7698146B2 (en) * 2002-04-24 2010-04-13 Volt Information Sciences Inc. System and method for collecting and providing resource rate information using resource profiling
US20040098265A1 (en) * 2002-11-05 2004-05-20 Sean Kelly Dialog management system
US20040107112A1 (en) * 2002-12-02 2004-06-03 Cotter Milton S. Employment center

Patent Citations (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5197004A (en) * 1989-05-08 1993-03-23 Resumix, Inc. Method and apparatus for automatic categorization of applicants from resumes
US5339239A (en) * 1989-10-13 1994-08-16 Mitsubishi Plastics Industries Limited Information collecting and/or service furnishing systems by which a user can request information from a central data base using a portable personal terminal and an access terminal
US5446919A (en) * 1990-02-20 1995-08-29 Wilkins; Jeff K. Communication system and method with demographically or psychographically defined audiences
US5416694A (en) * 1994-02-28 1995-05-16 Hughes Training, Inc. Computer-based data integration and management process for workforce planning and occupational readjustment
US5948040A (en) * 1994-06-24 1999-09-07 Delorme Publishing Co. Travel reservation information and planning system
US5717923A (en) * 1994-11-03 1998-02-10 Intel Corporation Method and apparatus for dynamically customizing electronic information to individual end users
US5754939A (en) * 1994-11-29 1998-05-19 Herz; Frederick S. M. System for generation of user profiles for a system for customized electronic identification of desirable objects
US5761662A (en) * 1994-12-20 1998-06-02 Sun Microsystems, Inc. Personalized information retrieval using user-defined profile
US5659731A (en) * 1995-06-19 1997-08-19 Dun & Bradstreet, Inc. Method for rating a match for a given entity found in a list of entities
US5832497A (en) * 1995-08-10 1998-11-03 Tmp Worldwide Inc. Electronic automated information exchange and management system
US5950200A (en) * 1997-01-24 1999-09-07 Gil S. Sudai Method and apparatus for detection of reciprocal interests or feelings and subsequent notification
US5978768A (en) * 1997-05-08 1999-11-02 Mcgovern; Robert J. Computerized job search system and method for posting and searching job openings via a computer network
US6266659B1 (en) * 1997-08-07 2001-07-24 Uday P. Nadkarni Skills database management system and method
US6213780B1 (en) * 1998-07-06 2001-04-10 Chi Fai Ho Computer-aided learning and counseling methods and apparatus for a job
US6089284A (en) * 1998-09-24 2000-07-18 Marconi Commerce Systems Inc. Preconditioning a fuel dispensing system using a transponder
US6289340B1 (en) * 1999-08-03 2001-09-11 Ixmatch, Inc. Consultant matching system and method for selecting candidates from a candidate pool by adjusting skill values
US6385620B1 (en) * 1999-08-16 2002-05-07 Psisearch,Llc System and method for the management of candidate recruiting information
US6406302B1 (en) * 2000-02-14 2002-06-18 Webb Nelson System and method for collecting and exchanging survey data
US6587832B1 (en) * 2000-03-16 2003-07-01 Compensate.Com Llc Market pay system
US6618734B1 (en) * 2000-07-20 2003-09-09 Spherion Assessment, Inc. Pre-employment screening and assessment interview process
US20020133502A1 (en) * 2001-01-05 2002-09-19 Rosenthal Richard Nelson Method and system for interactive collection of information
US20030145015A1 (en) * 2002-01-25 2003-07-31 Theodore Turnasella System and method providing user definable on-line wage and salary reports

Cited By (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7650286B1 (en) * 2003-04-18 2010-01-19 Algomod Technologies Corporation Recruitment vendor management system and method
US20040230989A1 (en) * 2003-05-16 2004-11-18 Macey William H. Method and apparatus for survey processing
WO2004104772A3 (en) * 2003-05-16 2006-09-28 Valtera Corp Method and apparatus for survey processing
US7418496B2 (en) 2003-05-16 2008-08-26 Personnel Research Associates, Inc. Method and apparatus for survey processing
US20080270218A1 (en) * 2004-05-11 2008-10-30 You Know ? Pty Ltd System and Method for Obtaining Pertinent Real-Time Survey Evidence
US20050268217A1 (en) * 2004-05-27 2005-12-01 Collegenet, Inc. System for describing the overlaying of electronic data onto an electronic image
US7917842B2 (en) * 2004-05-27 2011-03-29 Collegenet, Inc. System for describing the overlaying of electronic data onto an electronic image
US9608929B2 (en) 2005-03-22 2017-03-28 Live Nation Entertainment, Inc. System and method for dynamic queue management using queue protocols
US8447639B2 (en) 2005-03-22 2013-05-21 Ticketmaster Computer-implemented systems and methods for resource allocation
US7979291B2 (en) * 2005-03-22 2011-07-12 Ticketmaster Computer-implemented systems and methods for resource allocation
US8204770B2 (en) 2005-03-22 2012-06-19 Ticketmaster Computer-implemented systems and methods for resource allocation
US8517742B1 (en) * 2005-05-17 2013-08-27 American Express Travel Related Services Company, Inc. Labor resource testing system and method
US8103659B1 (en) * 2005-06-06 2012-01-24 A9.Com, Inc. Perspective-based item navigation
US9240013B2 (en) 2005-06-24 2016-01-19 Iii Holdings 1, Llc Evaluation of voice communications
US20110191106A1 (en) * 2005-06-24 2011-08-04 American Express Travel Related Services Company, Inc. Word recognition system and method for customer and employee assessment
US9053707B2 (en) * 2005-06-24 2015-06-09 Iii Holdings 1, Llc Evaluation of voice communications
US9530139B2 (en) 2005-06-24 2016-12-27 Iii Holdings 1, Llc Evaluation of voice communications
US8401884B1 (en) * 2005-11-07 2013-03-19 Avantas L.L.C. Electronic scheduling for work shifts
US20070266054A1 (en) * 2006-03-24 2007-11-15 Stephens Frank M Method and system for salary planning and performance management
US20150058240A1 (en) * 2007-04-30 2015-02-26 Experteer Gmbh Method and System for Providing a Virtual Job Market
US9477946B2 (en) * 2007-04-30 2016-10-25 Experteer Gmbh Method and system for providing a virtual job market
US11516200B2 (en) 2007-09-04 2022-11-29 Live Nation Entertainment, Inc. Controlled token distribution to protect against malicious data and resource access
US10715512B2 (en) 2007-09-04 2020-07-14 Live Nation Entertainment, Inc. Controlled token distribution to protect against malicious data and resource access
US10305881B2 (en) 2007-09-04 2019-05-28 Live Nation Entertainment, Inc. Controlled token distribution to protect against malicious data and resource access
US9912653B2 (en) 2007-09-04 2018-03-06 Live Nation Entertainment, Inc. Controlled token distribution to protect against malicious data and resource access
US20090077061A1 (en) * 2007-09-19 2009-03-19 Abercrombie Iii Charles Clinton Social network for travelers with layovers
US8341162B2 (en) * 2007-09-19 2012-12-25 Intercept, Llc Social network for travelers with layovers
US8751509B1 (en) 2007-09-19 2014-06-10 Intercept, Llc Social network for travelers with layovers
US9325755B1 (en) * 2007-09-19 2016-04-26 Intercept, Llc Social network for travelers with layovers
US8122371B1 (en) * 2007-12-21 2012-02-21 Amazon Technologies, Inc. Criteria-based structured ratings
US8875043B1 (en) 2007-12-21 2014-10-28 Amazon Technologies, Inc. Criteria-based structured ratings
US20210374769A1 (en) * 2008-03-05 2021-12-02 Ebay Inc. Method and apparatus for social network qualification systems
US20100082361A1 (en) * 2008-09-29 2010-04-01 Ingenix, Inc. Apparatus, System and Method for Predicting Attitudinal Segments
US10778730B2 (en) 2010-06-15 2020-09-15 Live Nation Entertainment, Inc. Establishing communication links using routing protocols
US9954907B2 (en) 2010-06-15 2018-04-24 Live Nation Entertainment, Inc. Establishing communication links using routing protocols
US10051018B2 (en) 2010-06-15 2018-08-14 Live Nation Entertainment, Inc. Establishing communication links using routing protocols
US10573084B2 (en) 2010-06-15 2020-02-25 Live Nation Entertainment, Inc. Generating augmented reality images using sensor and location data
US8676615B2 (en) 2010-06-15 2014-03-18 Ticketmaster Llc Methods and systems for computer aided event and venue setup and modeling and interactive maps
US9202180B2 (en) 2010-06-15 2015-12-01 Live Nation Entertainment, Inc. Methods and systems for computer aided event and venue setup and modeling and interactive maps
US11223660B2 (en) 2010-06-15 2022-01-11 Live Nation Entertainment, Inc. Establishing communication links using routing protocols
US9781170B2 (en) 2010-06-15 2017-10-03 Live Nation Entertainment, Inc. Establishing communication links using routing protocols
US11532131B2 (en) 2010-06-15 2022-12-20 Live Nation Entertainment, Inc. Generating augmented reality images using sensor and location data
US8977654B1 (en) * 2012-09-21 2015-03-10 Google Inc. Assigning classes to users of an online community
US9798815B1 (en) 2012-09-21 2017-10-24 Google Inc. Assigning classes to users of an online community

Also Published As

Publication number Publication date
EP1579364A1 (en) 2005-09-28
US20040210820A1 (en) 2004-10-21
US7571110B2 (en) 2009-08-04
AU2003300404A1 (en) 2004-07-29
US20090265224A1 (en) 2009-10-22
CN1754181A (en) 2006-03-29
WO2004061740A1 (en) 2004-07-22
WO2004061740B1 (en) 2004-10-07
EP1579364A4 (en) 2008-11-12

Similar Documents

Publication Publication Date Title
US7571110B2 (en) Automated compensation reports using online surveys and collaborative filtering
US7702621B2 (en) System and method for providing profile matching within an unstructured document
US20190050816A1 (en) System and method for evaluating job candidates
US20090132345A1 (en) Method and system for determining relevant matches based on attributes
US6915269B1 (en) System and method for facilitating bilateral and multilateral decision-making
US8036924B2 (en) System and method of identifying options for employment transfers across different industries
US20050060283A1 (en) Content management system for creating and maintaining a database of information utilizing user experiences
US20100070492A1 (en) System and method for resume verification and recruitment
US20140074560A1 (en) Advanced skill match and reputation management for workforces
US20130132284A1 (en) System And Method For Management And Deliberation Of Idea Groups
US20060085480A1 (en) Human resource sourcing exchange
US20060036603A1 (en) Apparatus, system, and methods for collaborative research
Marton et al. A question of quality: The effect of source quality on information seeking by women in IT professions
Botti et al. Goal ambiguity in public organizations: A systematic literature
Bexon et al. Competitive intelligence: a career opportunity for the information professional in industry
Tang The design and study of pedagogical paper recommendation
Hofmann et al. Integrating contextual factors into topic-centric retrieval models for finding similar experts
WO2007142941A2 (en) System and method for providing network source information
WO2024054800A1 (en) System and method for evaluating and scoring individuals and entities and displaying the score of each respective individual or entity
Duarte et al. Ordering Matters: An Experimental Study of Ranking Influence on Results Selection Behavior during Exploratory Search.
King " We should, but don't" paradox: Exploring user acceptance of a business intelligence system
Ryker Measuring user satisfaction as a function of the gap between users' perception of information systems' performance and user expectations
EP1671205A2 (en) Content management system for creating and maintaining a database of information utilizing user experiences
Patil et al. A Job Seeker’s Perception towards Online Job Portals
Srivastava et al. Rational versus institutional perspectives in organizational web sites

Legal Events

Date Code Title Description
AS Assignment

Owner name: PAYSCALE, INC., WASHINGTON

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TARR, DOUGLAS;GIORDANO, III, JOSEPH;REEL/FRAME:016394/0707

Effective date: 20050624

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION