Posts tagged with "AI"

Logitech Releases the Developer-Focused MX Keypad for AI Automation

Source: Logitech.

Source: Logitech.

Today, Logitech introduced the MX Keypad. If it looks familiar, that’s because it’s one-half of Logitech’s existing MX Creative Console. However, instead of bundling a 3 x 3 keypad with a dialpad, the MX Keypad is just the keypad for $99.

I reviewed the MX Creative Console in 2024, and I still use it and write about ways to automate tasks with it. It’s an excellent pair of devices, so it’s not surprising that Logitech has adapted the keypad to a new use case.

Keypad, cable, and kickstand. Source: Logitech.

Keypad, cable, and kickstand. Source: Logitech.

The MX Keypad’s hardware is identical to the keypad from the Creative Console. In addition to the keypad itself, it comes with a kickstand you can use to prop it up at an angle and connects via USB-C to your Mac. My MX Creative Console review has more detail about the hardware, along with images and a short video.

I like the MX Creative Console a lot, and according to Logitech, so did software developers looking for ways to automate their AI workflows. While Logitech didn’t expressly say why it dropped the dialpad for the MX Keypad, it makes sense to me having used the MX Creative Console for months. Dialpads are great for timeline-based creative work, but less so for development work. Yes, developers might want to use the dialpad to control screen brightness or volume, but by limiting the MX Keypad to just the keypad, Logitech was able to hit a lower price point, making the device more appealing to a bigger audience.

I’ve been impressed by the number of ways Logitech has opened its devices up to automators who like to tinker, too. Similar to devices like the Stream Deck, Logi Options+ lets you build Smart Actions based on keyboard shortcuts and macros. But there’s more than that. If you’d prefer to let someone else do the automation, Logitech runs a Smart Actions marketplace with plugins tailored to certain apps. For developers there’s also a Logi Actions API for building plugins with access to each Logitech device’s full capabilities.

Source: Logitech.

Source: Logitech.

From a developer standpoint, the combination of automation options means the MX Keypad opens up a variety of possibilities like:

  • launching agents,
  • monitoring agent activity via the device’s dynamically updated keys,
  • triggering saved prompts,
  • activating voice input, and
  • automating many other repetitive tasks.

Logitech is also teaming up with GitHub Copilot as part of the MX Keypad launch, similar to the partnership that was announced with Adobe as part of the MX Creative Console. There’s a Copilot plugin available that adds Copilot controls to the keypad’s keys. Logitech customers can also take advantage of GitHub’s offer for three months of GitHub Copilot Pro+ for free.

Logitech’s MX Keypad is primarily a repackaging of an existing device with a GitHub Copilot deal thrown in to make it a better value. But it’s good to see the company addressing the developer market more directly. Having used the MX Creative Console primarily for automation and development-type work myself, dropping the dialpad from the package to lower the price is a good tradeoff. I use the dial and its other buttons occasionally, but for automation, the keypad is where it’s at, and between Smart Actions and a full plugin SDK you can point your agent at, the MX Keypad is a great choice for automators.

The Logitech MX Keypad is available directly from Logitech and from Amazon.

Perplexity Introduces ‘Hybrid Compute’ for Computer Agent on Apple Silicon Macs

Source: Perplexity.

Source: Perplexity.

Perplexity has announced hybrid compute for Personal Computer on Apple silicon Macs. Personal Computer already combined cloud-based Computer tasks with access to local files and apps, but its AI processing was primarily cloud-based. Recently, Perplexity also introduced Portable Computer, a local-first version of Computer that can escalate work to the cloud with a user’s permission.

Hybrid compute approaches the problem from a different direction. Tasks start in the cloud but if a new local classifier detects potentially sensitive information, the user is alerted and asked whether they want to route it to a local model, although users also have the option to let the flagged data be used by a frontier cloud-based model. Gemma E4B, Qwen3.6 35B-A3B, and a Perplexity post-trained version of Qwen3.6 35B are available at launch for that local work, with more models to come in the future.

Tasks that don’t require sensitive information are handled in the cloud by frontier models that also orchestrate the cloud and local tasks. Users select the cloud orchestrator and Perplexity chooses the helper frontier models used.

Another key feature is that hybrid compute works with tasks begun from an iPhone or iPad, too. Those devices could already kick off a session, but now they can also run tasks that use sensitive information locally if your Mac is turned on and running Perplexity.

Yesterday, I got a demo of the feature that walked through three scenarios:

  • an attorney researching a legal brief, with sensitive client data on their Mac, allowing them to update it locally,
  • an analyst using web-based data to update a confidential slide deck for a presentation about a company’s financials, and
  • a small business owner kicking off pricing research from her iPhone that was combined with private pricing data for her business that never left her Mac.

I haven’t tried hybrid compute myself yet, but if it works as advertised, it’s a big step forward for Perplexity, and one that a lot of its competitors are also chasing. AI agents are only as good as the tools and data to which they have access. Tool calling has been a big focus over the last year and more, but data privacy is an equally important issue that needs to be solved before more companies, especially those in regulated industries, adopt agentic tools like Personal Computer. It’s good to see Perplexity tackling this head-on.

Inside OpenAI’s Codex with Andrew Ambrosino→

On today’s episode of AppStories, Federico and I interview Andrew Ambrosino, the project lead for Codex. It’s a great conversation that covers a lot of ground, including:

  • the long history and many iterations of Codex,
  • the merging of Codex into a new version of ChatGPT based on web technologies,
  • Computer Use and Computer History, which came along with OpenAI’s acquisition of Sky last year, and
  • what goes into designing for user trust.

Then for AppStories+ subscribers, we dug into Andrew’s Mac origin story and some of his favorite Mac apps.

AppStories is available on Apple Podcasts and all your favorite podcast players, as well as YouTube. AppStories+ is a special ad-free, extended version of AppStories that’s delivered a day early in high-bitrate audio for $5/month or $50/year by itself or as part of a Club MacStories Premier membership that adds our weekly and monthly newsletters, app discounts, and other perks for $12/month or $120/year.

Permalink

Parker Ortolani’s First Impressions of the Pebble Index AI Smart Ring→

Parker Ortolani on the Pebble Index, the recently released AI smart ring:

Ambient computing is here at home on speakers and in the workplace with apps like Wispr Flow, it’s only a matter of time before we carry it with us everywhere.

The Pebble Index is the best preview of that future. The simple gesture of pressing your thumb to your index finger is unbelievably natural. Especially for quick interactions. I’ve found myself setting more reminders, recording ideas, and keeping lists up-to-date. And I know that everything is being logged on my phone in the Pebble app. Better yet, you can connect the Pebble Index to your other AI tools using MCP. There’s also no reason for skeptics to fret, it’s not always listening for hot words or logging everything it hears.

As Ortolani explains, the device ticks a lot of boxes that other AI gadgets just don’t:

The thing that I think strikes me hardest with the Index is that it’s just so socially inoffensive. It’s a form factor people already love, it’s not distracting, it’s not always listening, and it’s affordable. It could be the first truly noncontroversial AI-first hardware.

I ordered a Pebble Index months ago and it should be coming within the next week or so. I bought it for many of the features Ortolani mentions. It’s on-demand, discreet, and a form factor that doesn’t look out of place. The hardware and software matter a lot with a device like this, but if Pebble has nailed it, I can see it becoming my go-to way to jot down brief notes. I’m excited to give it a try and report back.

Permalink

Anthropic Introduces an In-App Browser for Claude Cowork

Hot on the heels of OpenAI’s addition of a cloud-based browser to ChatGPT Work, Anthropic has added an in-app browser to Claude Cowork in its desktop app. What’s interesting about the two browser implementations is that while both ChatGPT Work and Claude Cowork are meant to be used for general knowledge work, Claude Cowork’s browser is something of a hybrid that mixes aspects of OpenAI’s Codex built-in browser with its ChatGPT Work cloud-based browser.

First of all, I do not yet have access to Claude’s in-app browser, so the following is based solely on Anthropic’s announcement and other documentation. Still, the differences are worth understanding since the tools are aimed at different workflows.

Like Codex’s in-app browser, Claude Cowork’s in-app browser is built into the desktop app itself. That ties Claude’s browser to the instance of its app running on your Mac. Turn your Mac off or close the Claude app, and the browser becomes unavailable. Leave the Claude app running, and you’ll be able to control the browser session remotely through the Claude mobile app or the web.

From there, however, the browsers diverge. As you’d expect, the Codex implementation is developer-oriented, with tools to build, inspect, and annotate web pages along with access to a terminal, repos, and diffs. In contrast, Claude Cowork’s in-app browser opens automatically as needed for general-purpose web tasks instead of development tasks.

Also, while both ChatGPT Work’s and Claude Cowork’s browser implementations benefit from being isolated from your other day-to-day browser use outside those apps, their handling of login credentials is a little different. As I wrote in greater detail yesterday, ChatGPT separates the login process from the model’s activity and evaluates the login page for things like phishing schemes. Claude Cowork’s in-app browser offers two ways to log in: manually, or by importing cookies from logged-in sessions in another browser. Currently, Claude supports cookie imports on the Mac from Chrome, Edge, and Firefox, but not Safari. In addition to the security benefits that come with a browser that runs isolated from your everyday browser, Claude also requests permission before acting on a site for the first time, blocks high-risk sites, checks actions against the your requests, and uses the same prompt injection protections as its Chrome extension.

My first-run experience with ChatGPT Work’s cloud browser was a mixed bag but showed promise. Although I haven’t tried Claude Cowork’s browser yet, I’m glad to see Anthropic moving in the same direction as OpenAI here. Isolating agent browsing by bringing the browser into Claude and ChatGPT doesn’t solve every security issue, but it’s a good starting point. I expect we’ll see both Anthropic and OpenAI iterate quickly on this theme in the coming months.

Hands-On with ChatGPT Work’s New Cloud Browser Feature

Yesterday, OpenAI introduced a new ChatGPT Work feature designed to let it navigate websites that require a login without revealing your credentials to the model.

Like a lot of people, I spend far too much time clicking around websites that require a login, looking at analytics and other data, checking whether new sponsors have been booked for our podcasts, and more. It’s a tedious but necessary part of my week that slows me down and takes me away from writing and other creative work. ChatGPT’s new Work feature is designed to handle that sort of work for you.

The feature works by signing into websites using a virtual, cloud-based computer. That separates the browsing session from any browsing session on your local computer, walling the agent’s computer use off from your open tabs, cookies, browsing history, passwords, and other data. Because the login and browsing happen in the cloud, that also means work can continue whether you close the ChatGPT app or power down your computer.

According to OpenAI, an additional review model checks the sign-in request and credential destination for signs of phishing or deception. ChatGPT Work then pauses while the user signs in. Credentials entered through its secure sign-in form go directly to the cloud browser and are not visible to the model. After the user authenticates, ChatGPT resumes its work.

OpenAI also explains that its cloud-based browser doesn’t store your username and password. Instead, it saves cookies that allow you to return to a previously authenticated session. If you don’t want to remain logged in, ChatGPT’s cloud browser settings allow you to clear browser data for individual websites or all sites.

Similar to other features in ChatGPT and Codex, users have choices when it comes to which sites ChatGPT Work can access. “Always ask” is the default, requiring the agent to check with the user before using a website, but the permission level can also be set to “Auto approve” after ChatGPT checks a site for relevancy and risk or “Always allow,” which OpenAI discourages. Individual websites can also be allowed or blocked. These settings control website access, but consequential actions require separate confirmation.

ChatGPT Work's browser control works best on a mobile device and with simple login systems.

ChatGPT Work’s browser control works best on a mobile device and with simple login systems.

I gave ChatGPT’s browser use a try and the results were mixed. I wasn’t able to get the feature to work at all using ChatGPT Work in Safari on my Mac. Some websites had security measures in place that prevented me from logging in. Other times, the cloud browser asked me to take over manually, but I was unable to do so because the UI was frozen or login panels didn’t appear.

Logging into a site with a CAPTCHA required manual intervention.

Logging into a site with a CAPTCHA required manual intervention.

I had better luck using my iPhone. On one site with a simple username and password system, a login sheet appeared. I entered my credentials, was logged in, and the agent navigated the site to answer my queries. On Apple’s affiliate link dashboard website, I had to take over the browser interaction manually to satisfy a CAPTCHA, which worked after several challenges. Once logged in, the agent pulled and analyzed the data I requested. Also, after I’d signed in to both of those sites on my iPhone, I remained logged in to ChatGPT’s cloud browser, which meant I could also use them from my Mac.

In its current form, the idea of having ChatGPT Work browse signed-in websites on your behalf is better than its implementation. If you can get logged in, having an agent collect and analyze things like analytics data is fantastic. However, getting past the initial login screen is still too frustrating. That said, I’ll be keeping a close eye on the feature, which is available on eligible plans depending on rollout and workspace settings, for future use collecting and analyzing data that would otherwise require a lot of clicking around.

The Potential of M6 and M5 Ultra for Local AI on macOS

Earlier today, Apple unveiled the new generation of Mac mini and Mac Studio, featuring the latest entries in the Apple silicon family of chips: the M6, available in the Mac mini, and the M5 Ultra, exclusive to the Mac Studio. You can read more details about the announcement and related specs in John’s overview.

As someone who’s been working with an M3 Ultra Mac Studio (on loan from Apple) with 512 GB of RAM for the past year (plus two separate M4 Mac minis) with a particular focus on local AI models and performance gains enabled by Apple’s MLX framework, I obviously am very interested in these new machines. To give you some context: my current workspace for my upcoming iOS and iPadOS 27 review, which lives in Notion, is entirely managed by a series of local agents running the latest DeepSeek-V4-Flash via MLX on the Mac Studio, which continuously scan the project for new notes, sources, and research material that is automatically categorized and linked in the chapters that I’m writing. So, yes, I’m sure I’ll have more thoughts on the new Mac minis and Mac Studios soon. In the meantime, I thought it’d be fun to break down the local AI-related details from Apple.

Read more

Defining an “Agent Harness”→

I’ve recently been asked to explain what an “agent harness” is (particularly after I wrote about my favorite app of the year so far, Open Minis). I realized it was one of those concepts I could understand intuitively but not quite articulate. Thankfully, other people have done a better job of explaining it than I have.

Drew Breunig calls them “situated agents”:

Harrison Chase once excitedly shared an insight that agents are comprised of 4 things: a system prompt, a planning tool, a file system, and subagents. In the year-plus since he said that, I think this remains largely true. (Though you might tweak it to have general tools, etc.)

[…]

Imagine the simplest coding agent, and you at the keys. Let’s slowly zoom out and consider all the elements the harness can manage:

  1. The Session: The current task and context, as both a trajectory and a durable, branchable log. You can zoom backwards, fork, and replay it.
  2. The Environment: The instance, defined as a sandbox, terminal, worktree, computer, and/or container.
  3. The Repo: The project. Versioned with Git, it contains your code, history, current work, AGENTS.md, guides, and hooks.
  4. Memory: The person’s predilections, accrued over time, managing progress and past decisions.
  5. Skills: The domain, artifacts describing reusable workflows or domain knowledge worth wielding in this situation.
  6. The Team: Your colleagues and counterparts. Shared rooms, shared traces, project tracking tools, issues, and bug reports.
    • The Organization: The policies and audits, defined by legal, leadership, and procurement.
    • The Model: The LLMs themselves, the common artifact shared by all. Stochastic blobs we all poke trying to evoke positive outcomes. Log or train on their quirks and adjust.

I also thoroughly enjoyed this explanation and excellent visualization by Ted Spare, Dexter Storey, and Sarim Malik, writing for Rubric Labs:

A harness is the software that translates a model into a system that can affect its environment.

[…]

Functionally, a harness makes a model agentic, meaning it can take action.

And:

As coding agents begin to run for longer, and deploy more intelligence through dispatch, they are owning larger and more complex problems end to end, and the developer is less in the loop to steer the agent. The value of high quality planning increases as agents implement the plans more autonomously. Harnesses like Claude Code and Codex ship with a native planning mode, where the agent must first create a detailed Plan.md file with feedback from the user before executing. These harnesses then place a reference to the plan and the todo list into a top level state (system prompt) so that the agent doesn’t forget what it’s working on across long runs.

Given the multi-model, hybrid structure of the new Siri AI, we should probably assume Apple also made a lightweight “Siri harness” to aid the on-device orchestration of the entire system.

Permalink

Apple Music to Launch Labels on AI “Songs”→

Ethan Millman, writing for The Hollywood Reporter:

Apple Music will soon launch labels on songs created with artificial intelligence, the company said in an email sent out to industry partners on Thursday.

In the email, obtained by The Hollywood Reporter, the streaming service said that AI music labels will come “later this year,” though Apple Music didn’t disclose a specific launch date.

The new labels come months after Apple Music launched transparency tags back in March, available for record labels and music distributors to disclose when content uploaded was “materially generated using AI.”

Good. As much as I enjoy working and tinkering with AI, AI “music” falls squarely in the category of AI implementations I do not understand and cannot even be remotely interested in. Perhaps I’m old-fashioned, but what’s the point of listening to “music” performed by something you can’t see live on a stage, or that you know has never lived a real life? (Obviously we can’t see The Beatles or Beethoven perform live today, but we know they were real humans with real motivations behind their work.)

If you ask me, AI music shouldn’t even be allowed on streaming services, but I suppose it’s too late to fix that problem by now. Hopefully Apple will also include a system-level toggle to permanently hide AI “music” from Apple Music’s UI, too.

Permalink