In my previous blog what is driving the innovation in nlp and gpt3<\/a> , I talked about how GPT3 has evolved from the basic transformer architecture.<\/p>\n Based on that blog, a start-up approached me saying that they had an idea which they felt could only<\/strong> be implemented by GPT3.<\/p>\n They were eagerly waiting to be approved (isn’t everybody – he he!)<\/p>\n Apart from waiting for GPT3- there was another critical flaw in their argument<\/p>\n Their idea was not generative i.e. it did not need GPT3 in the first place (or for that matter any similar architecture)<\/p>\n It’s tempting to think that GPT-3 will solve all the NLP problems .. but it does not<\/strong><\/p>\n let me explain by this what I mean by this<\/p>\n Below is the basic flow of NLP services and a listing of NLP applications<\/p>\n NLP services include:<\/p>\n Image source – Dr Amita Kapoor<\/a><\/p>\n While many of these are generative- not all of them are.<\/p>\n The GPT3 and transformer-based applications basically address the generative elements of NLP<\/p>\n That still leaves a large number of other applications which use NLP but are not generative (for example Text classification or Text summarization).<\/p>\n <\/p>\n You can also look at the same situation from the perspective of word embeddings<\/strong>. Word embeddings are a type of word representation that allows words with similar meaning to have a similar representation.<\/p>\n Historically, word2vec and GloVe have worked well for word embeddings but these were shallow approaches. Transformers solve this problem by providing a functionality similar to what we see in transfer learning for CNNs (thereby not all layers need to be trained if you use a pre-built model)<\/p>\n <\/p>\n To conclude<\/strong><\/p>\n Hence, we can say that GPT3 is very interesting and will continue to be so.<\/p>\n However, there will be always a subset of NLP applications which will not be covered by any of the transformer-based approaches because they are not generative.<\/p>\n <\/p>\n<\/div>\n Go to Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"\n
<\/a><\/p>\n