While engineering our prompts well can maximise the possibility of a good result from GenAI tools that we are using, it is also important to understand the technology, the features that AI are really good at, and the things that they are really poor at so that you’re leveraging the right strengths and not wasting your tokens or prompting attempts on things that are fundamentally not going to come out of the AI tool.
One thing to appreciate, especially for GenAI, is that LLMs are designed to provide you with high probability responses that will be considered ‘right’ or ‘appropriate’ by human raters. So even though they are trained on a large amount of data and information, it is difficult to necessarily weight the information appropriately to your needs when giving you a response. As a result, they would tend to give you motherhood statements that have a high probability of being on-point even if they are tangentially related to what you are really looking for. You may be able to guide it in your prompt to go a bit more specific towards what you want, but essentially, you’ll have to do the work of narrowing down things. This is because the LLM is designed not to clarify your questions and sharpen their output for you – that’s something that you are supposed to do. Yet at the same time, they would already be consuming your tokens even during that process while you’re ‘finetuning’ them towards your output.
The other challenge for LLM is the problem of hallucination and even creating false data or connections. Quite often even when they are citing or linking to certain sources, the LLMs are making guesses on the content and association with what you’re searching for and also what they are ‘talking about’ themselves. So an assertion may be associated with a link they furnish you, rather than being based on that link. This is radically different from academic citations when your ‘sources’ are really saying or validating what you are saying. Most GenAI tools simply provide materials that may be associated rather than actually make certain points. Worse still, they could make up links that are broken and claim them as source.
Of course, there’s a whole issue around existing softwares being replaced by AI or vibe-coding. Often, this involves almost reinventing the wheel. Yes, maybe the prototyping cost has come down: what AI has done is that it has taken down some of the initial barriers in getting some kind of digital product out. But all that without providing a proper long-term infrastructure planning or system thinking because it is not exactly optimising towards a longer-term vision. This means a lot more resources dealing with bugs and bolting on new features or other aspects of the software in a way that is not optimised at the system level. Moreover, the AI companies are themselves competing with their own customers and users in developing the more bespoke tools for those willing-to-pay clients. So what makes you think you are going to vibe-code your way to a proper product people will pay you for?