Source: Huoxun Finance
Article reprinted source: AIcore
Original source: Deep Thinking SenseAI
The change of technology will affect the information collection and distribution mechanism. After the invention of printing, people used indexes and directories to find books, and after the advent of the Internet, people used keywords to find links. The PageRank algorithm invented by Larry Page, the founder of Google, and the optimization of algorithm strategies such as intention recognition are aimed at helping users find better web links through the task-based distribution mechanism centered on the algorithm.
AI makes information search no longer a one-way matching of keywords and links, but an intuitive and accurate two-way dialogue. Perplexity AI first turned this vision into reality, with a one-year valuation of 500 million US dollars. It is a conversational search engine created by former Open AI employees, which gives people a glimpse that a search is not only about discovery, but also about the future of understanding.Based on the development history of search under the technological change, this paper deeply interprets the current AI+ search head product Perplexity AI, and looks at the boundary between search and search engine.
Product priorityPerplexity used a lot of APIs to build products in the early stage. The team focused on product-level optimization, made up for the potential problems caused by the lack of model capabilities through deep product insight, and then went to self-developed infrastructure to reduce costs.
Information interactionPerplexity is a good example of using LLM to assist interaction with external information under the condition of minimizing hallucinations. This form is not limited to search engines, and any scene that needs to interact with external information can expect the reconstruction of LLM.
The boundary of searchThe migration and closure of content by platforms such as Little Red Book and WeChat official account limited the development of global search engines. However, the search itself used to rely on the search subject to collect, filter, summarize and integrate a large amount of information. Now AI can externalize the search process interactively. Search engine has become a new content platform.
AI Native product analysis
Perplexity AI
1. Product: Perplexity AI
2. Product launch time:December 2022
3. Founder:
-Aravind Srinivas：CEO, studied for a PhD at UCB, focusing on reinforcement learning and image recognition. During my Ph.D., I worked as an intern in OpenAI, DeepMind and Google. After 21 years of graduation, I joined OpenAI to study language models and diffusion models.
-Denis Yarats：CTO, worked as a machine learning engineer in Quora, and studied reinforcement learning, optimal control and robots in Meta AI Institute.
-Andy Konwinski: Co-founder and co-founder of Databricks.
-Johnny Ho: Chief Strategy Officer, formerly a quantitative trader
4. Product introduction:
Perplexity is a Swiss army knife used for information discovery and curiosity satisfaction. It helps users summarize content, explore new topics and stimulate creativity by answering questions.
5. Development history
-In August 2022, Srinivas founded Perplexity after leaving OpenAI.
-In September, 2022, it received $3.1 million in seed round financing.
-In November 2022, ChatGPT was launched.
-In December 2022, Perplexity AI was launched.
-In March 2023, Perplexity raised $25.6 million in the A round of financing, with a valuation of $150 million.
-In October 2023, the ARR reached USD 3 million after the subscription service was launched, and a new round of financing led by IVP was completed, with a valuation of USD 500 million.
01. The evolution of search
The development of search can be traced back to people’s demand for information and the evolution of search and distribution methods.
The concept of search can be traced back to the emergence of printing, when people began to use catalogues and indexes to help them find books and documents. In the mid-1990s, Yahoo! Early search engines began to appear, and they used keyword matching to help users find web pages, but the search results were not always accurate or complete.
Google was founded in 1998 and was invented by Larry Page, one of its founders. PageRank algorithm, by evaluating the number and quality of links between web pages, ranks the weight and importance of a given web page. After that, website owners need to understand PageRank algorithm and other search engine ranking factors in order to optimize the website to improve its ranking in search engines.
Search engine optimization (SEO) experts began to study the selection and use of keywords, the optimization of website content, the construction of external links, the improvement of user experience and other strategies to adapt to the development of Google. They strive to make the website get higher visibility and ranking in the search engine results page (SERP) through technical means and optimization methods.
Referring to our previous article, Greylock, a top venture capitalist: a high-potential market that will be reshaped by AI, Greylock proposed that AI would turn the original algorithm-centered information distribution network into a decentralized two-way network between people. As far as search is concerned, search is expected to become a bilateral information interaction instead of unilateral information query and SEO strategy optimization.
02. what is perplexity ai?
Perplexity is almost the earliest generative search engine, or answer engine. Borrowing the power of the big model, users can ask questions directly, and Perplexity will directly summarize from various screened sources, provide accurate and direct answers, and provide source reference at the same time.
When ChatGPT was first introduced, it once made people think that generative AI might replace the traditional search engine with its excellent natural language understanding and the ability to generate rich answers. However, with the problems of illusion in user experience, inability to connect to the Internet and lagging knowledge update gradually emerging, people began to return to reality and turned to search engines enhanced by large models, such as Perplexity and Bing Chat. These "answer engines" use RAG (Retrieval Augmented Generation) technology to process the results of search engines, so as to reduce misleading information and improve the timeliness of information. In addition to Perplexity and Bing Chat, other platforms that tried to challenge the dominance of Google search engines, such as You.com and Neeva, also turned to the AI-enhanced answer generation mode.
Compared with the traditional search engine, the answer engine is optimized in the following aspects:The ability to understand user questions, summarize search results, retain the index of search results, and expand user questions.These optimizations are aimed at lowering the user’s use threshold, saving the time for users to search and browse on different web pages, ensuring the reliability of search results, and providing users with the ability to dig deep into problems.
It is precisely because of these characteristics that when Bing Chat was officially launched in February this year, Microsoft CEO Nadella placed high hopes on it and regarded it as an important symbol of opening a new era of search. He believes that this is an unprecedented challenge to Google’s dominant position in the search engine market for 20 years. However, by the time of Google’s antitrust case in October, Nadella showed a change, and frankly Bing Chat still had many problems to be solved, and it did not achieve the expected results in the market share competition. On a global scale, the pattern of search engine market remains stable.
Although Perplexity is still far behind the mainstream search engines in terms of visits, and there are many criticisms that it is only superficial packaging, since its launch, Perplexity has maintained a steady growth, maintaining the highest access time among similar products, and its performance even exceeds that of You.com, which has many years of experience in AI combined with search engines. Among the top 50 GenAI products released by a16z, PerplexityAI ranked 10th. In the half year from March to October, the number of search requests handled by Perplexity AI increased by 6 ~ 7 times every day, and now it has to handle millions of search requests every day.
Perplexity has maintained a good development momentum even after other large-scale models have introduced networking capabilities. Therefore, this paper will discuss the uniqueness of GPT "shell" product Perplexity in the eyes of many people, combining with the actual use experience.
03. How does Perplexity stand out?
Three advantages of Perplexity: fast iteration speed, good effect and functional innovation.
Iteration speed is fast, with a small update every week.
As shown in the figure below, since the introduction of Perplexity, its important updates and milestones are clear at a glance. In less than a year, Perplexity has achieved several key version iterations. Especially in the period of frequent updates, new functions are introduced almost every week, and these updates mainly focus on product functions. It was not until October this year that Perplexity launched a large-scale model aimed at reducing operating costs.
By analyzing the details and data related to version updates filtered out by Perplexity in Twitter, including the release date of each version and the number of likes of related tweets, it can be seen that the Twitter search engine launched at the end of last year was the most liked. The team has a keen product insight into search, and launched Twitter’s SQL-based search analysis engine half a month after the launch of GPT-3.5. The second is the new model launching class dynamics, while the Perplexity team pays more attention to and puts less attention to the iterative content of product update.
It can be speculated that at present, the public’s cognition of AI+ search products is still in the early stage of similar products, and has not yet entered the in-depth experience of product functions or formed significant user stickiness.
Functional innovation, complement the shortcomings of the answer engine
"Devil in the details.", Perplexity’s excellent search experience benefits from its many innovative functions, especially Source Edit, Focus Search, and Perplexity Copy..
Perplexity doesn’t always perform well. For example, when querying "Who is the CEO of Twitter", although similar products can correctly answer Linda Yaccarino, Perplexity sometimes answers incorrectly. This error stems from its reference to Wikipedia entries that have not been updated in time. For such errors, the Source Edit function can provide an effective solution.
Source Edit allows users to edit reference sources and search again. At present, this function only supports deleting rather than adding sources, which effectively reduces the interference of irrelevant sources on the results and corrects the potential instability manually. It can be seen that after excluding Wikipedia, which contains erroneous information, Perplexity can give the correct answer.
In addition, users can use the Focus Search function to limit the search scope before starting a new search, thus improving the search effect. This function is specially optimized in academic search, mathematical calculation, YouTube video and Reddit forum search. Especially YouTube video search, its reference can directly link to the exact time point of related content in the video.
Perplexity Copilot enhances the accuracy and credibility of search results. As a user’s search assistant, Copilot provides more detailed, in-depth and personalized answers.
For the same question, Copilot Search usually refers to more sources, longer answers and more structured presentation. At the same time, in the search process, Copilot will extend the meaning of the user’s question. In a user’s search, in fact, it has searched for different keywords many times. As shown in the figure below, using Copilot to search for the same keyword, Copilot will automatically extend the user’s intention, use different keywords to search and finally summarize.
Personalized search. Perplexity Copilot not only deeply understands the user’s intention, but also provides customized content according to the user’s personal situation. For example, when asking for restaurant recommendation, users will be automatically asked to supplement necessary information, such as the location of the restaurant; At the same time, Copilot will supplement the information required by the user’s AI Profile. As can be seen in the following figure (right), after the author has set his own city in the AI Profile in advance, Perplexity Copilot will no longer ask the user to supplement the address information. Finally, when Copilot asks the user for supplementary information, it will adopt a more LLM Native interaction mode, and according to the type of supplementary information required, Copilot will choose the most appropriate interaction mode for the user to input, as shown in the following figure (right), and a set of check boxes will be automatically generated.
Perplexity copy uses Fine-tuned GPT-3.5 instead of GPT-4. According to the test, Fine-tuned GPT-3.5 can provide the same or even better performance as GPT-4 in most cases (69%), and even provide better performance than GPT-4 in a few problems.
Perplexity’s vision is not only to become a better search engine, but to build a comprehensive knowledge center to help users learn new knowledge easily.For this reason, Perplexity has been focusing on optimizing its citation source and divergence problem handling ability since the beginning of development.
In September, Perplexity launched the "Collections" function around this vision. In Perplexity, each query conversation is regarded as a Thread, and collections are containers of threads, which function like favorites. Collection can not only organize threads, but also expand new problems around the theme and invite collaborators to build a knowledge community together.
Excellent effect, fast, accurate and reliable.
Perplexity shows excellent performance in many aspects, especially in the reliability of content, the richness of information sources, the rapidity of response and the stability of content.
First of all, the reliability of its content and the richness of its sources are particularly remarkable. Taking searching for the latest Dev Day update of OpenAI as an example, searching in Bard, Perplexity, GPT4 and You.com shows the most comprehensive source citation and the best search results in both Chinese and English. Although GPT4 relies on correct keywords, the quality of its results is closely followed, while Bard and You.com are not comprehensive in both Chinese and English searches.
In addition, Perplexity also performs well in the stability and generation speed of results. Compared with other competitors, Perplexity can basically provide consistent answers based on the same source when querying the same question repeatedly, which effectively reduces the uncertainty of the big model. At the same time, the speed of generating answers is the fastest among all similar products.
In addition to qualitative analysis from the perspective of user experience, some scholars try to quantitatively evaluate the effect of the answer engine. In April this year, Nelson F. Liu of Stanford published the paper "Evaluating Verification in Generative Search Engines", and evaluated several major search engines such as Bing Chat, Perplexity, YouChat and Neeva. This paper tests from four dimensions: text fluency, perceived validity, citation recall rate and citation accuracy. On the whole, Perplexity performed the best in this evaluation.
04. Perplexity AI’s experience is insufficient
Based on the above advantages, some users think that Perplexity AI can completely replace the traditional search engine; However, some users have given completely opposite opinions, thinking that it is difficult to replace traditional search engines because of its insufficient information sources and low result value.
The gap in evaluation may be due to the inadaptability to the high expectations and usage habits of the answer engine.
High expectation value
Nelson F. Liu, the author of the paper Evaluating Verification in Generative Search Engines, thinks that on the whole, such answering engines are far from their expected performance. The existing answer engines usually produce smooth and informative results, but they are not good at citation recall and citation accuracy-only 51% of the generated statements are fully supported by the citation content, and only 74.5% of the citations can correctly support the generated results.
The author also found that the generated content has obvious negative correlation in citation accuracy and validity. This is probably a price to reduce the illusion, which is reflected in the fact that the generated result is often a direct copy of the quoted content or a Paraphrase of the quoted content. When the source is actually irrelevant to the user’s problem, this problem will be very obvious.
For example, when users ask, "Is cooperation or competition the driving force to guide social evolution?" Sometimes, the answer engine may refer to the content of cooperation and competition in animal evolution. At the same time, the author thinks that the research results can not fully evaluate the effect of the answer engine, becauseThe research focuses on the verifiability of the answer engine results, not the practicality.The emphasis is on the secondary verification of the reference value and the accuracy of the reference. It is assumed that the user will use the reference to verify the search results based on the generated results. However, what users expect is to get the answer directly without secondary verification, and this expectation often fails, because the answer engine is good at summarizing and not good at stitching.
For most factual questions, Perplexity AI performs very well, and can complete the search target without external links, such as searching for the update of OpenAI Dev Day, the launch of SpaceX rocket, the raiders of a temple in the legend of zelda, etc. The characteristic of this kind of question is that if you use the traditional search method, you can usually get the answer by opening a single web page, and the answer engine refers to multiple sources to summarize the most important information and shorten the description.
But sometimes, the generated results will give people a sense of being reasonable, but useless. The reason for not finding really useful information may be that the density of this part of information is extremely low, and even if AI reads all the data, it cannot extract the essence efficiently.Those things that are often mentioned are also more likely to be things that AI thinks are important and learned first.If the current autoregressive model of transfomer seems to be logical, frequent high-frequency information will increase the probability of being predicted as the next word. "When the answer given by Perplexity is not more meaningful than what I already know, Perplexity acts as if it can’t find the answer or the answer doesn’t exist at all, but when searching on Google, I can still find something deeper that I really want in the first page".
In a word, users expect different granularity answers to different search questions, but Perplexity AI can’t fully consider this when giving answers. The product itself tries to overcome this problem by giving options such as changing models, introducing Copilot and editing search keywords, but it still needs to be optimized for a long time.
Different usage habits
SEO(Search Engine Optimization) has been studied for a long time. It is a process to improve the ranking of websites in search engine results by optimizing website content. There are some interesting statistics about search:
1. 69.6% of the search keywords are less than 4 words in length.
2. 65% of the time, users will select the page to jump from the search results within 10 seconds.
3. In 25.6% cases, users will not click on any search results.
4. Less than 1% of users will browse the second page of search results.
5. About 59% users can solve the problem with one click.
Note: Statistical results refer only to orders of magnitude.
In most search scenarios, simple keywords are used to filter from search results in a very short time, and the problem is solved in less than or equal to one page. This habit is not suitable when it is transferred to the answer engine, because this kind of answer engine search relies on accurate description of the question, prompt engineering and multiple rounds of dialogue to optimize the results; At the same time, it will take longer to generate results than traditional search engines. Perplexity AI weakens this problem by displaying the source before generating the results, but it is still slower than the traditional method.
High expectation value and different ways of using it will lead to the quick search problem that can be solved by opening a page in traditional search, and the answer engine needs more detailed search conditions and longer time to get similar results; For complex search problems that rely on cross-validation of multiple pages, the answer engine appears to be incomplete in information source or insufficient in ability, and cannot give enough valuable information, so users need to use traditional search engines to search again.
05. Can you subvert the search engine?
Although the answer engine is highly anticipated, it is still a distant goal to subvert the traditional search engine.
Take Neeva, which was established in 2019, as an example. This company was once one of the potential challengers of Google, and invested a lot of resources to build its own indexing and ranking system, aiming to provide a better user experience than Google, and without advertisements. Neeva was close to Google’s level in user research and internal indicators, but it closed its business for ordinary consumers only two years later because of the lack of enough users. This shows that,It is difficult to change the pattern of search engine market only by optimizing user experience.
In the past, search engines built deep barriers. Nadella, CEO of Microsoft, and Ramaswami, co-founder of Neeva, both said that the search engine is the most difficult market to break through in the Internet, and most users will not change their default search engine. At the same time, the more users use the default search engine (Google), the search results may be continuously optimized through a large amount of user data, and its leading position seems to be hard to shake.
In addition to the barriers established by traditional search engines, answer engines have other common problems to solve based on big models, such as cost and feedback mechanism.
At present, most of the answer engines are based on traditional search engines and APIs of large models, and they are more focused on the optimization of models and RAG (retrieval-augmented generation). At the same time, the answer engine products generally have not found a reasonable business model. Many competitors’ products of Perplexity AI are still completely free, and it is difficult to meet the large demand for API only by the income of subscription mechanism, which prevents the possibility that the answer engine completely replaces the traditional search engine at this stage. At present,Perplexity AI has been trying to build its own WebCrawler, Search index and LLM to cope with the growing number of query requests and reduce costs.
In addition, the answer engine has failed to find a product form that integrates the collected user feedback into the normal use of the product, just like the search engine or information flow recommendation, so it is difficult to build a data flywheel based on the first-Mover advantage and user accumulation. According to CEO Aravind Srinivas, only about 10% of users will provide feedback, and users may not like the results generated by AI for various reasons. Just because users don’t like it doesn’t mean that the results are not well generated. It is likely that some users’ desired results are lost in the summary of AI. As mentioned earlier, users’ expectations for the summary are not stable and consistent, and what AI thinks is good is not necessarily what humans think. At this point, unlike Google or Tiktok, Perplexity AI can’t use every click or like of users to further optimize products, and still rely on some external Contractor to annotate user data to further optimize the model.
06. More than just search engines
The real threat to search engines may not be another tool, but the centralized migration and closure of content.
In the past ten years, the content published by users has migrated from open forums and blogs that support search engine indexes well to platforms that do not support search engine indexes so well, such as WeChat official account, Little Red Book, Tik Tok, Instagram, Twitter and even Amazon. These platforms have become the first choice for many people in certain scenarios. In the future, these ecosystems are likely to have their own AI search assistants. Not only content platforms, knowledge management platforms such as Feishu and Notion, but also plans to launch their own AI search assistants. The experience and usage methods are similar to those of the answer engine, but the search scope focuses on private databases.
In any scenario, AI-driven search is an assistant for users to interact with external information. Ideally,AI-driven search will help users have an intuitive two-way conversation with information, rather than one-way matching based on keywords.
In addition to the model’s ability to understand and process information, the size of the database will also be one of the important factors that limit the ceiling of answer engine products. The impact of closed platforms such as content platforms on search engines will also have an impact on the future of answer engines. How to build a unique database or access more external databases may be the development focus beyond the product and model capabilities. For example, can Rewind AI be regarded as a search product based on user’s screen recording data to some extent?
Perplexity AI’s team is also aware of the barriers of search engines and the trend of content changes, so they did not choose to make products in the vertical direction to compete, such as shopping assistants or life assistants, because they could not compete with Amazon and Tiktok in data, but chose to explore in the direction of becoming a knowledge content platform.
Try to build a content community by allowing users to share and save the multi-round dialogue process between themselves and Perplexity AI; Although AI currently performs well in expression ability, A large number of seemingly reliable but not practical contents produced by AI limit its potential as a knowledge platform.
But the good news is that search engines are still in a large number of scenarios to help users solve practical problems, and search engines and answer engines can still provide reliable value. It is often said that search itself is a kind of ability, which needs to collect, filter, summarize and integrate a large amount of information, and the processing of information is in the past search process, which exists in the workflow of the search subject but cannot be spread, and more is processed and processed in the brain and personal knowledge base of the search subject. With AI replacing users to complete the processing and display of information, the search process itself has the potential to become an interactive content form, which may be the possibility that Perplexity AI will become a content platform.
reference material
https://youtu.be/ix4_rdogcVI