Although I will not be treated in general machine learning or artificial intelligence textbooks etc, I will summarize what I regard as important personally in data analysis. The text in "" is quoted. I will gradually organize quotations and others.
Caution. Translation from Japanese to English uses Google Translate.
Philosophy of data analysis
*wrap up:
As a way to capture the essence of things, philosophy has history. Understanding such wisdom of mankind is important as it becomes a shortcut to understand and solve problems. Philosophy is the cornerstone of scientific thinking. *
- The biggest difference between philosophy and religion methods is that religion explains the world by story (= myth), while philosophy uses abstract concepts to do this, universality across ethnic groups and nations It is in the point that it is intended.
- The method of philosophy began with the concept of "fundamental" thinking and "universal" thought.
- "Universalization" is to be established as a language game which makes it possible to exchange ideas with the outside of the community.
- Requirements for having philosophical thinking. One is abstract thought ability. Another thing is "thought of" universality ". And the third is free consciousness.
- A disadvantage of philosophy is that unified views on "principle" are difficult to establish.
Causality and Correlation
History of philosophy of causality
Plato
*wrap up:
The causal relation in the real world is a network structure. Among them, what is the "causal node" depends on the human subjectivity and the choice of sensors to recognize and define the node. And it is determined by the value of "What is the best?" To solve problems. *
"The philosophy that" transcendental entity "(transcendent) exists. The founder of European thought of such a philosophy is Plato, and its finished person was regarded as Hegel "
"According to critics, Plato and Hegel are the greatest representatives of the European" intelligence "symbolized by the words" truth "," universal recognition "," absolute "
"In other words, as a crowd, there is not much positive evaluation for Plato.
"Plato introduced the problem of exploring the principle of" soul "for the first time into the thought of Greek philosophy that had been pursuing the" cause "and" principle "of the world as nature,
"Connecting the problem of conventional nature recognition to problems of human life and value such as spirit, soul, virtue, true, good, beauty, for the first time it is the foundation of the philosophy (philosophy) as the study of human exploration Philosopher "
"It can be said that the Platonic world image is a world image that makes the modern naturalistic, rationalistic world image stand up.
"Another criticism is the criticism of so-called anti-Platonism (anti-truth, anti-universal, anti-transcendent) from modern thought that I mentioned before, which is ideally a big power now"
Heidegger says "The idea theory is not criticized by modern rationalism, but rather the idea theory is considered to be the root cause of" bad "modern rationalism.
"Plato was the founder of the essential meaning of the philosophy in Europe (philosophy) The essence of philosophical thinking is that it is the thought of" principle "and the thought of" universality "
"The central idea of Plato is the idea theory, as I have seen, often it is first regarded as the highest principle of the world order that exists in the world of Higan,
"When Achilles reached the point where it was previously crushed, the turtle will always be running a short distance before Achilles, no matter how small it is, this circulation will continue logically indefinitely.
"Infinite things" are not included in "finite ones". This is the proposition that supports the whole paradox of Zeno, but this is nothing obvious. "
"When drawing a circle, its inside becomes a" finite "area. However, arbitrary "points" can exist indefinitely within this finite inner region. "
"We say" finite> infinity "as" contradiction "In our head" logic order "is" wrong "
"Abstract concepts such as" infinity "and" finite "are just one" point of view "to be established each time"
"Descartes 'God' s Proof of God" is also exactly the "sophistry theory" which is made up of this "conceptualization of conceptual image"
"The misunderstanding of" realization of concept "became an essential depression for philosophical thinking with abandoning the story and using abstract concept as a rule. "
"By the way, Hobbes, Hume, Kant, Hegel, Nietzsche, etc are hardly depressed in this trap.In modern philosophy, in particular Husserl and Wittgenstein are clearly aware of this problem.
"Philosophical thinking is not only the effort to explore the" principle "but also the task of doing this work against the hollowing out of logic by instantiation of concepts at the same time. "
"Socrates appears as a first-handed awareness about this subject in philosophy.
"Does existence like" God "exist? (= There is the root cause of the world?), This is an interesting question that anyone is interested in somewhat. However, to tell the truth, this question is "there is no answer". "
"Philosophically, this question is clear that" there is no answer "(not understanding the answer). It is Kant that we first revealed that there is no answer to this question in principle. "
"In the first place, the answer in the series" cause - result "comes out in two areas, that is, in the area of experience and in the area of pure logic. "
"The world of purely logical reasoning has no" in principle "inevitable relationship with the world of experience, the former originally originated from the latter, but the two are just alike. "
"In principle pure inference is not established in principle in what sort of origin of the world, God exists, what happens when it dies, etc. So these are questions that do not answer in the first place. "
"The phraseology is the real way of philosophy to continue thinking deeply about the question" What is this existence of myself? ", Just like literary literary hobbies are always literature haunting always stuck in any age . "
"This pretense" metaphysical question "is a big factor that makes us lose sight of the essence of philosophy, alongside the challenges of" paradox "as we saw earlier"
"The pretense question of" metaphysics "comes from mistakes trying to apply reasoning which only occurs in the field of pure logic to the human experience world as it is. "
"The essence of what is called the" cause / basis "of various orders can not be regarded as a logical" causal relationship "that is realistic or displays it. "
"Because it creates arbitrary viewpoints and thus creates arbitrary lines, it creates many" causes "and it does not go anywhere for that.
"The true" cause / basis "of various orders should rather be essential to the" valuable relationship "that is value-worth what is" good ""
"The core of the transformation of this" cause "concept can be said to be" internalization "of the concept of" cause "and" basis ". "
"That is, it means that the concept of" cause "has its essence in the power of the meaning of the subject (spirit) that captures it rather than the causal relationship of" things "and" words "itself Means "
"What is inherently inquisitive for human beings is only what is the best in terms of things"
Immanuel Kant
*wrap up:
In response to the question "Do you have a godlike existence?" Kant proved that "there is no answer" (although some people continue thinking afterwards). If so, it means that human beings have greatly reduced the time to think about useless things. In that sense I think it 's amazing. But how? *
"If it is a rationally recognized ability and the reason that should be the highest decision body of truth is becoming a hotbed of its own peculiarity, this is the problem specific to philosophy science attached to unquestion"
"What Kant discovered is nothing less than a peculiar inherent in this reason.
"Kant 's unique idea of reason, distinct from empirical hypotheses such as sensory and optical illusion - called" transcendentalistic "
"Transcendentalistic" is a concept that is the opposite of "empirical" here and refers to things that do not have any corresponding sensory data. "
"A sarcastic phenomenon that as a result reasoning advances inference based on the law of its own reasons, it reaches a conclusion contrary to reason"
Origin of Criticism of Reason "He confesses that he used an interesting method," I proved one proposition and at the same time proved the opposite proposition ". "
"The pair of contradictory propositions (one pair), Kant named it" antinomy "or" antinomy. " "
"Of the set of propositional pairs forming antinomy, the affirmative proposition is called" thesese "and the negative proposition is called" antithesis ". "
"Paradox has been one of the greatest concerns of logic since the latter half of the nineteenth century, but not much is pointed out, but the origin of the problem in modern times is Kant 's antinomy.
The philosophy of God and others is not directly found in the experienced world but is indirectly thought out by "reasoning". There is a pitfall where reason is self contradiction. "
An example of antinomy "Thesis: There are absolutely inevitable existence in the series of world causes Antithesis: There is no absolutely inevitable existence in this series, where everything is coincidental. "
"Since the same reason is proving simultaneously two propositions contradictory, the truth of both propositions is lost.
"Because the reason that should be the supreme determination authority of the truth has caused such a situation, doubt is put on the reasonability itself.
"If there is size, it should be either a finite amount or an infinite amount, but for the world as a whole, the assertion that it has a finite amount (thesis) also has an infinite amount Antithesis) were both judged as fake.
"By concurrently proving the thesis and antithesis in which the same rationale contradict each other, as a result, the world does not actually exist.
"The shock that Hume gave to Kant was criticism of its causality.The causal law (also called causality) is a law that connects cause and result, that is, the appropriate result is due to an appropriate cause It is a rule.
"According to Hume, causality is not an a priori objective law, it is only a subjective association of ideas, a product of mere association.
Kant clearly proves that causality is one of the a priori fundamental laws that enable our empirical recognition.
"When causality as the champion of reasonable awareness traces back to the first cause, it immediately loses the effect and meaning that it was proud of until then, or it can be said that the causality law fails in the face of the problem of the first cause.
"When considering causality as a mere associative law, he merely self-naming the law by replacing subjective things with objective ones, which is only a hypothesis (a hypothesis) "
"In philosophy, the definition is not a starting point, but rather an end point to be a goal, and philosophy has many proofs that can not be proven, in particular all of principle's truth can not be proven"
"Kant philosophy was a match between fundamental concepts that can not be defined and fundamental truth that can not be proven from the very beginning, even though it is called later criticism philosophy or criticism of reasons"
"Proof is a procedure that provides a basis for some thing, that thing that can be given a basis can not be a basis for itself, so it is not the root of things.
"The fundamentals of things can base things, but there is no reason to base them on any further grounds because it is principle's truth because there is no grounds for going back any further.
"Therefore, principle truth is impossible to prove in principle.
"In the face of something that can not be proven in principle, meaning that you can not help confining the way of proof"
"All fundamental concepts and fundamental principles that Kant was able to do in criticism of reasoning are to be proof in principle in principle.
Scientific thinking method
*wrap up:
Scientific thought in modern times is based on deductive method and induction law. Both have drawbacks, but have a complementary relationship. *
"Science can generally be defined as" legal knowledge proven based on empirical methods such as observation and experimentation "
Modern science can be paraphrased as a combination of argumential science based on deductive method and experimental science based on induction law, or rational and empirical methods.
"First make a hypothesis explaining unknown natural phenomena, derive observable consequences from that hypothesis, and experimentally verify the consequences"
"Academic methodology is a procedure to demonstrate the correctness of guided conclusions in a form that does not allow anyone to admit, thinking about things that are problematic"
Argument is "to derive a single proposition to conclude according to a certain rule (inference rule) from a premise group of propositions"
"Knowledge is expressed linguistically in the form of" sentence ", and a meaningful sentence whose true or false is clearly defined is called" proposition ". Therefore, it is the proposition that constitutes the assumption and conclusion of argument. "
"Deduction" is a method of logically deriving individual propositions (conclusions) from a universal proposition (premise) "
"The characteristic of the deductive method is that the premise (axiom) conclusively (without exception) conclusions (theorem) consequences, therefore, if the premise is correct, the conclusion is always right.
"However, the conclusion is that which is already explicitly included in the premise explicitly, it can not expand knowledge by deductive method, that is, acquire new knowledge.
"Inductive" is a proof that derives a universal proposition (conclusion) from an individual proposition (premise). Contrary to the deductive method, inductive methods can expand knowledge, but it can be assumed that The relationship with the conclusion is not inevitable, it will remain stochastic (stochastic).
"A number of philosophers have challenged as a problem of" justification of induction "to base the correctness of induction as a method of argument, but unfortunately none of them succeeded. "
"Deductive methods and induction methods have their own strengths and weaknesses, it is no other than hypothetical deductive method to try to make up for the shortcomings by taking advantage of the advantages of both.
Obtaining data
We will summarize the know-how of collecting classical information in intelligence activities and not scraping and API.
Intelligence activity
Humint
"Huminto is a technique to recruit appropriate people in other countries as agents (information sources) and to provide confidential information"
Osinto
"Osinto (open source intelligence) is an information activity that gathers and analyzes public information such as newspaper and magazine articles carefully and explores confidential information, but more than 90% of confidential information excluding military information can be obtained in Osinto It is said.
"According to Richard Friedman, who served as analyst at the CIA, the most common sources of information were the Parliament Library and a paid database such as LexisNexis," Reuters "," The New York Times "
Spy and Diplomacy
"For intelligence for life it can be understood that it is the ability to sort out various information (information) around himself.
Wiretapping
hacking
Forecast
control
The theory of controllable
Creativity
Enhance creativity at the individual level
# # Brain and creativity
Enhance creativity at the social level
Concept of "anti-fragile"
Unsolvable problem
Cause
Structural change
outlier
Forecast affects future state
How to respond
Find similar problems
Find and use irrational behavior
Expand the time frame, space frame to observe
theoretically to solve
Trap that engineers tend to be addicted
- If you do not use technology for their own benefit, you can not go ahead. When it comes to developing for the benefit of someone, it will be left to development that matches the understanding of that "someone".
- Do not overstate "cutting edge algorithm", "data volume", "machine power". It is difficult to solve the problem properly if you do not have imagination to the causal relationship of the event that is happening.