Claude, Perplexity, Microsoft Copilot, Gemini, ChatGPT, You... or... WHAT?

Claude, Perplexity, Microsfot Copilot, Gemini, ChatGPT, You.com?

What Generative AI model and tool should you choose?

I can understand how overwhelming this may be. In fact, it is for me, too. There more I learn about Generative AI the more tools I discover and explore.

The market is flooded with tools that do music, videos, images, content, dating, programming, narration and everything in between. And at least when it comes to choosing an Generative AI assistant, such as Claude or Copilot or ChatGPT, things can get a little bit confusing.

You can avoid reading the rest of this article by just doing this: experiment. Create a free account in all the models and tools, experiment with them and test the quality of the output, and choose what works for you. However, if you want to save some time, continue reading for some tips.

Please note that I created these tips thinking of you as an individual person, not a organization. Some of the insights shared in this post may apply for an organization looking to implement AI solutions, but they are more valuable for you as an individual person.

Dimensions and Tests

Here are the top 10 categories I would recommend considering when assessing which AI model or tool to use, especially in the context of HR. In addition, I included three ways in which you can test the models or tools in each the dimensions in order to form a thorough and informed perspective about which tool can help you best.

Conversational Abilities: How natural, contextual, and human-like the AI's responses feel in a back-and-forth dialogue.

• Ask open-ended questions on various topics and see if the AI's responses stay coherent and on-topic

• Introduce abrupt changes of context mid-conversation to test if the AI can adjust naturally

• Try using slang, idioms, or playful banter to gauge the AI's understanding of conversational variations

Knowledge Breadth: The range of topics the AI seems knowledgeable about beyond just factual recall.

• Quiz the AI on facts across diverse domains like history, science, arts, current events etc.

• Ask for in-depth explanations on complex topics to reveal the depth of the AI's understanding

• See if the AI can make insightful connections between disparate concepts

Creative Proficiency: The AI's aptitude for open-ended tasks requiring imagination, such as ideation, storytelling, or humor.

• Request the AI to generate original stories, poems, songs or other creative works

• Ask for innovative ideas or solutions to open-ended prompts

• Evaluate the AI's ability to combine concepts in novel, imaginative ways

Analytical Skills: The depth of analytical reasoning and problem-solving capabilities the AI exhibits.

• Present the AI with complex scenarios or data sets and ask for an analysis

• Request step-by-step explanations for how the AI arrives at conclusions

• Assess the AI's ability to break down problems, weigh tradeoffs, and suggest solutions

Emotional Intelligence: The AI's ability to perceive, understand, and respond appropriately to emotional tones and social cues.

• Use prompts expressing various emotions and see if the AI mirrors the tone appropriately

• Ask the AI to rewrite texts while adjusting for different emotional perspectives

• Evaluate the AI's understanding of emotional subtext and social dynamics

Personalization: How well the AI can tailor its personality, communication style, and responses to the individual's preferences.

• Note if the AI adjusts its language, tone, and phrasing to your personal communication style

• See if the AI can maintain consistency in the persona or traits you assign to it

• Check if the AI can adhere to specific preferences or customization instructions you provide

Trustworthiness: The sense of reliability, honesty, and ethical grounding the AI projects in its outputs.

• Ask the AI to fact-check claims or highlight potential biases or inconsistencies

• Probe the AI on ethical dilemmas to gauge its moral reasoning abilities

• Note if the AI gives straightforward acknowledgments when it lacks knowledge on a topic

Learning Pace: How quickly the AI seems to accumulate new knowledge and adapt to the user's inputs over time.

• Provide the AI with new information and see how rapidly it can utilize that data

• Note if the AI's outputs get more tailored and contextually relevant over an extended conversation

• Check if the AI can quickly infer and apply rules or patterns based on examples

User-Friendliness: The intuitiveness and accessibility of the interface for communicating with the AI.

• Assess how easy it is to understand and use the AI's input prompts and output formats

• Note if the interface provides helpful examples, tips or explanations to guide users

• Check for accessibility features like voice input/output for improved user experience

Cost and Access: The financial and technical barriers to using the AI for an individual's needs.

• Research pricing models, subscription fees, pay-per-use costs etc. for the AI

• Check if free trials or free tiers are available to try out the AI

• Look into technical requirements like compatible devices, internet connectivity etc.

Limitations and Risks

I “trust” these tools, but not blindly. I don’t share personal or corporate data. In addition, if you can, create your account using your personal email (nothing wrong with using a corporate email, but just in case).

AI models and tools are amazing and powerful. But as “smart” as they may sound, they lack true comprehension and deeper human reasoning abilities. They excel at pattern recognition and generating coherent responses based on their training data, but cannot genuinely understand context the way humans do. This lack of deeper understanding can sometimes lead to nonsensical or biased outputs, especially on complex topics the AI hasn't adequately learned. That’s why you have to be patient, design great frameworks for prompting and, more importantly, add the human touch. Remember: “AI won’t replace your job, but a human using AI will”.

AI is only concerned with outputs, and you should be concerned with outcomes. The output is the end-result of your input and AI help. But, the outcome, the impact you are trying to achieve, is optimized by taking AI’s output and adding the human touch and element to it. That’s the kind of collaboration you should keep in mind.

5 Ways to Create a Testing Case

Finally, here are some ideas to truly assess an AI model's capabilities across conversational abilities, knowledge breadth, creativity, analytical skills, and more, it's important to test them with complex, real, kind-of-human, scenarios that mimic real-world HR situations.

Use “Real” Employee Data

Create “real”, but anonymized employee date. For example, feed the AI authentic details about job roles, performance, grievances etc. to evaluate its context comprehension.

Craft Multi-Layered Narratives

Don't just use simple prompts - build intricate narratives that interweave different HR aspects like managerial relations, policy conflicts, cultural tensions etc. Introduce emotional components, ethical dilemmas, and shifting contexts to test the AI's ability to navigate complexity.

Incorporate Multimedia

Go beyond just text by including audio clips, images, videos, data visualizations and more in your testing cases. See how well the AI can perceive and integrate information across different mediums. Not all models allow for multimedia inputs, but test the ones that do.

Crowdsource Edge Cases

Crowdsource unique challenges from a diverse group of HR professionals, managers, and employees across your organization or peers in your network. Their personal experiences can uncover edge cases that stress-test the AI's reasoning abilities. Do the exercise together with them. Try to fry AI’s brain to see how far you can stretch its capabilities, and which of the available ones is… the one (for you).

Iterate and Evolve

Don't just evaluate once - create an iterative process where you continuously evolve your testing cases based on the AI's performance. Identify areas where it struggles and introduce new complexities to push the boundaries of its capabilities.

By Enrique Rubio (he/him)

(Note: I write fast, speak fast and read fast. Sometimes - often times - I don't see the typos! Please let me know if there are typos or grammatical misconstructions in my write ups. It happened to Shakespeare, it certainly can happen to me. Do so with kindness, compassion and grace, though. I appreciate it!)

Coming Soon